Journal posts (2020)

I have been using Dropbox for a long time, possibly for 10 years or so. Over time I've been putting more and more of my home directory into it, since it's so convenient. At some point it grew to 100 gigabytes and was connected to 7 or 8 devices (not counting the API-only Android file managers). But over that time, I also saw the app and the service grow into a bloated mess instead of the simple sync tool that it once was.

Zawinski's law says that every program attempts to expand until it can read mail. Similarly, the fate of every cloud service is to morph into an enterprisey "collaboration tool" with features nobody asked for. Signing in to the Dropbox website immediately greets you with Paper, Apps, Spaces, Showcases, and a tiny area in the middle where your actual files are. (The only thing that isn't there is Photos, which was useful so they removed it.)

This all would be fine if it had been confined to just the web UI, but at least on Windows it wasn't. This blog post illustrates it well – the desktop client was a mess of promos, popups, notifications, and remember when it grew its own file manager app to replace Explorer? The last drop was when they started annoying me with deliberately non-dismissable "Dropbox Family" promos as if I wasn't giving them enough money already for Plus.

The Linux version fared better, since most likely none of the higher-ups cared about Linux, but on the downside, none of the higher-ups cared about Linux. The daemon worked without X11 (which is better than the Nextcloud client), but only if you knew where to download the separately-distributed CLI tool from. It would crash if you started it via nice. It would deliberately nuke its configuration if the inode number was different (like if you restored it from backups), and it would nuke itself for no reason if you were using XFS, and instead of fixing the issue they just made it refuse to run on anything but Ext4. (Though I literally just found out now that they began accepting ZFS/XFS/Btrfs again in v77, which is okay I guess.)

Don't get me wrong, aside from all those problems it was a useful tool that worked transparently most of the time, and they did work on things like the new sync engine (which greatly improved certain things on all operating systems), but more and more it felt like it just barely met the expectations. (On my server, I had to put ~/Dropbox on an ext4 loop image!) And at some point I had already started using Unison and Syncthing for a few other things – e.g. several GB of document scans which only needed to be synced between two computers – not to mention git-annex, and began really wondering if I could reduce just the amount of different sync tools in use.

At first I moved ~/Music to Syncthing from Unison – yes, I do keep all my favourite songs locally. Seeing it working well (most of the time), I then added a few "config" folders that previously had been symlinked into Dropbox. Now, a year later, I grew tired of running both side-by-side and went full in, first adding ~/Dropbox into Syncthing on my server, then gradually doing the same on every computer (pointing it at the existing files to avoid having to re-clone the entire 100 GB).

This did require some untangling of mysteriously desynchronized states, but overall it went well, and two weeks ago I cancelled my Dropbox subscription and completely removed it from all my machines. I've hit some new minor problems since then but still I have no intention of going back.

The good

• It's cheaper. Actually, buying just the HDDs for my server has already cost me as much as three years' worth of Dropbox Plus, but worth it. (Not to mention I can put other things in there, like backups or git-annex.)
• Multiple sync directories. Instead of having to symlink things like FFXIV settings into ~\Dropbox, Syncthing lets me directly define the location I want to sync and it doesn't clutter my regular files – the additional directories are opt-in, rather than being extra bloat I would need to manually exclude on every host that doesn't need them. And for that matter, I don't need to have the main directory located specifically at ~\Dropbox, although I've decided to keep it at that location to satisfy muscle memory.
• The daemon uses less resouces and starts up faster. After bootup, the initial rescan takes just a few seconds, while Dropbox would sometimes take minutes. (Especially noticeable on the old dual-booting Raindows laptop, where I've seen Dropbox spend tens of minutes and always thought it was Windows 10 at fault, as it wasn't nearly that bad when booted into Linux.)
• Files with non-BMP characters (e.g. newer emojis) can be synchronized, which Dropbox couldn't do. Modify timestamps are synchronized, which Dropbox didn't do. Symlinks stay as symlinks, which Dropbox could do but only since last month or so.
• Windows Explorer is fast now. Although the Dropbox shell extension did a kinda useful thing with its icon overlays, it was amazingly slow, and made Explorer feel sluggish even on an SSD. Now it's really snappy. (And I don't really miss the overlays much – I didn't even have them on Linux at all, most of the time I looked at the global sync state instead.)
• It's open-source. Also, it's not cloud, it's not collaborative, it's not business, and it's not other buzzwords.
• No more "oh you must use ext4 only" bullshit.

The bad (and the meh)

There are, however, some issues with it – primarily with the core metadata sync engine that Syncthing uses.

Syncthing: 13 items in this directory are out of sync
Me: Which items?
Syncthing: None (page 1 of 2)
Me: Maybe you would like to go and sync them?
Syncthing: No

Sometimes, rapid changes can semi-permanently desync at least the displayed state. For example, my server still thinks my laptop is "out of sync" due to a foo.jpg.crdownload file (which Chrome renamed to foo.jpg long ago). On a few occassions I have seen a local folder being in "error" state because it couldn't pull some files which had already been deleted from the source, and it was because the local Syncthing received the batch of file additions; queued the downloads; and didn't process a later batch of deletions until it was done with the downloads. Whenever that happens, the only way to unwedge it is to create dummy files at those paths again.

Also:

• Case-only renames on Windows. While Dropbox ignored those outright, Syncthing would get really confused when it received such a rename. I think this was finally resolved in version 1.8.0 but I haven't yet dared to test.
• Directory renames. Dropbox just learned this recently with its new sync engine (Nucleus), but Syncthing is still unaware of renames so it has to reindex and sync the directory contents in full. If you're lucky, other hosts will pull the new file before the removal, doing a local copy – but often they'll delete the old file first and end up needing to re-download the whole thing from you.
• Configuration affecting peer devices can take a real while to apply. For example, if I share a folder with device B, I'd expect device B to notify me within a minute – but in reality it doesn't show up unless I click "Pause" and "Resume" (forcing a complete disconnect and reconnect) on device A.
• Instant file monitoring sometimes stops reacting to events entirely, ignoring all file changes until I manually click the "Rescan" button. At first I thought it was a Windows problem but it occurs on Linux as well.
• Syncthing doesn't have a kernel driver for on-demand files. Although Dropbox never had this functionality on Linux either, it was marginally useful on Windows e.g. for my ton of PDFs in "Library".

There's still quite a bit of community around Gopher sites. It was a way to publish information before the Web took over, and it seems quite a few people in the "tilde-verse" still develop Gopher clients, maintain personal sites and publish "phlogs" via gopher:// today.

Gopher might be a way to escape the fancy ReactJS-based, resource-hogging web of today, but unfortunately it goes well into the opposite extreme – it's rather clumsy and only marginally better than hosting .txt files over anonymous-FTP (because indeed text files arranged in directories is about all you get).

But it turns out the world isn't standing still and they are building a new thing called Gemini, which aims to be "somewhere in the middle" between HTML and raw text. It's a whole new protocol which serves pages in a whole new markup format, but it's kept deliberately very simple. (You get some markup, but it's even more minimal than Markdown. You get hyperlinks, but they're not inline with text.) The main site of Gemini project is of course served via Gemini, but just as with Gopher and Finger, there are web gateways.

I actually kind of like the idea, and the various clients (at least in the screenshots) seem a bit more welcoming; they look like something you'd actually want to read articles in, rather than feeling like a dusty library catalog. Looking at the Motif-themed Castor screenshots actually reminds me a bit of VNs like lost memories dot net that are set in the late-Geocities era of the web – though in the case of Gemini, the style is all client-side.

* * *

Gemini always uses TLS. It's not a protocol meant for retrocomputing, and compatibility with obsolete operating systems is explicitly stated as its non-goal. I think that's a good choice for any new project.

I still deliberately don't impose HTTPS on the main website. Although it does fully support HTTPS (with TLSv1.2 enforced), it still remains accessible without, because compatibility with obsolete operating systems is a goal of this website. (Or in other words, because I'm vain and I try to load it on whichever old OS I've got installed.) For those of you reading this through Internet Explorer 5, a CDF channel is available!

There are various websites which offer to check your own domain for known issues, and they sometimes give you a single-letter grade. I found one recently, entered my own domain and got immediately graded as "Fatal error". Turns out, although most things were alright, I committed the mortal sin of not having an HTTPS redirect on port 80.

(The check-tool also scolded me for wasting my visitors' bandwidth by using inline <style> here and there, and by not doing GZip. Sorry.)

You probably know that Linux can look up user accounts from the traditional /etc/passwd file, or from Sun's NIS/YP system, or from an LDAP directory server (if you install additional software). But did you know the Linux glibc also comes with built-in support for retrieving user account information in DNS?

Here's an article about Hesiod, a mostly forgotten service that's still present on every Linux glibc system to this day.