How Dropbox Rewrote the Heart of Its Sync Engine
A practical account of why Dropbox rewrote its desktop sync engine in Rust, with a rewrite checklist and hard-won lessons about data models and testing.
The oldest rule in software says you never rewrite a core system from scratch. Dropbox spent four years doing exactly that to the code every user trusts with their files — and opens the story by admitting the move would have been terrible in most environments.
Source
Rewriting the heart of our sync engine
By: Dropbox (Sujay Jayakar)
Published: Mar 09 2020
Type: Engineering article
I saved this because most rewrite retrospectives either celebrate or warn, and this one does both with its reasoning shown. It turns a four-year bet into a decision checklist — the specific conditions that made a rewrite right for Dropbox and would make it wrong almost anywhere else.
Why Dropbox rewrote the sync engine anyway
A sync engine is the client-side code behind the “magic folder”: it watches a directory on your computer, uploads your changes, downloads everyone else’s, and keeps the server and every device agreeing. Dropbox shipped the first version in 2008, when sync meant one person’s files on a few machines. By this article it meant company sharing hierarchies holding millions of files, more content than any laptop can store — Smart Sync exists to fetch files only when opened — across hundreds of billions of files, trillions of revisions, and exabytes of customer data on hundreds of millions of devices.
The article’s first job is making you feel why that scale breaks sync, through four stacked problems.
First, this is a strange distributed system. Most distributed algorithms treat a network partition as an anomaly to recover from; here, clients going offline for long stretches and reconciling later is normal operation. The old engine represented a move as two operations — a delete at the old location and an add at the new one — so a transient network hiccup could apply the delete and drop the add. A file the user had simply renamed would then vanish from the server and every other device.
Second, durability has to hold everywhere. Windows, macOS, and Linux each run a zoo of filesystems, kernel extensions, and hardware, and applications lean on behavior that was never actually guaranteed. Rare filesystem bugs only show up in large populations, so at scale “just works on every machine” and strong durability guarantees pull in opposite directions.
Third, testing is combinatorial. A shared folder can have thousands of members, each with a differently out-of-date view, different pending uploads, and different partial downloads — and syncing a single file already interleaves parallel chunk transfers, disk writes, and reads. The number of possible system snapshots times the number of valid action sequences is astronomical.
Fourth, sometimes “correct” isn’t even defined. Their example: Alberto and Beatrice edit offline; he moves Archives into January while she moves Drafts into Archives. Replayed directly, the folder graph contains a cycle — Archives inside January inside Drafts inside Archives. The old engine escaped by duplicating directories; the new one keeps the originals and lets upload order decide. Three folders and two moves already defeat naive intuition.
So why rewrite? By 2016 the old engine looked solved: hundreds of millions of users, years of production hardening, a team of sync experts. Joel Spolsky had called rewriting from scratch the “single worst strategic mistake that any software company can make.” Yet every behavior change still required an arduous rollout, complex inconsistencies still surfaced in production, onboarding new engineers took years, and performance work never raised the ceiling on how many files the engine could manage.
They had genuinely tried the incremental path — cleaning up ugly code, refactoring interfaces, adding Python type annotations, building telemetry and safe-maintenance processes. It wasn’t enough, and the article is precise about why. The data model was designed for a simpler world without sharing: files had no stable identifier that survived moves, and consistency guarantees were few. You can’t change the foundational nouns of a system in small pieces. The engine also wasn’t built for testing — its permissive model allowed states that were technically legal but wrong, and you can’t assert correctness against a model that permits garbage. And its threading handed scheduling decisions to the operating system, making integration tests non-reproducible; the team coped with coarse locks, trading parallelism for sanity.
That analysis becomes the reusable part of the post: a rewrite checklist in three questions. Have you exhausted incremental improvements — real refactoring, profiler-driven hotspot work, shipping intermediate value along the way? Can you pull it off — do you deeply understand the old system (they call it doing archaeology), do you have the engineering-hours and the domain experts, and can you accept slower feature development? And do you know what you’re going towards — a concrete reason the second system will be better, plus principles written down early? Dropbox’s principles were testing, correctness, and debuggability, encoded directly in the data model.
What they built, codenamed Nucleus, earns that setup. It’s written in Rust, which they call a force multiplier — less for speed than because the team could encode invariants in the type system and let the compiler check them. An invariant is just a rule that must always hold, like “a file’s identifier never changes when the file moves.”
Nearly all the code runs on a single thread, the Control thread, using Rust’s futures to interleave many concurrent actions without operating-system threads. Network IO goes to an event loop, hashing to a thread pool, filesystem IO to a dedicated thread. The payoff isn’t simplicity for its own sake: with inputs and scheduling fixed, the Control thread is deterministic, so they fuzz it with seeded, randomized simulation — random filesystem states, schedules, and perturbations — and any failure can be replayed exactly from its seed. Millions of scenarios run every day.
The client-server protocol was redesigned around strong consistency too. The client and server must agree on the state of the remote filesystem before a mutation counts as done, shared items carry globally unique identifiers and never appear duplicated or missing in transit, moves are atomic regardless of subtree size, and any discrepancy between the two views is by definition a bug.
My take
The reframe I’d offer: strip out the Rust and this is a testability story. The two decisions that made the new engine survivable — stable identifiers in the data model, and a deterministic core you can replay — are available in any language. Rust’s real job is enforcement: keeping those invariants from quietly eroding over the next decade of changes. That makes it the third most important decision in the post, not the first.
Read that way, the checklist is really testing whether your failure lives in the nouns or the adjectives of the system. Tangled modules and slow hotspots are adjectives, and incremental work fixes them. Wrong identifiers, missing invariants, and unreproducible tests are nouns, and no refactor reaches them. Dropbox could name the noun-level failure precisely, which is what turned the rewrite from hope into a rational bet.
The caveat is one the article supplies itself: they had domain experts doing archaeology on the old system, an organization able to slow feature development for years, and a business reason pulling forward. That’s the exception the old rule describes, not its refutation.
Where this is useful
- Deciding refactor-versus-rewrite on a core system: run the three checklist questions in order, and stop at the first honest “no.”
- Designing data models that can be tested: stable identifiers that survive moves and renames, and invariants strict enough that “legal but wrong” states cannot exist.
- Building concurrent systems that need reproducible tests: a deterministic single-threaded core with IO offloaded beats a fleet of OS threads you can’t replay.
- Not applicable when the incremental list isn’t exhausted yet, or when nobody on the team can do archaeology on the old system — that rewrite starts from ignorance.
Apply this: Pick the system you’re most tempted to rewrite and write down the incremental fixes you haven’t tried: module refactors, profiler hotspots, shipping intermediate value. If that list isn’t empty, the rewrite can wait. If it is, name the data-model change no refactor can reach — that sentence is your actual business case.
What I want to remember
Remember: A rewrite is justified by the nouns, not the adjectives. When the failure lives in the data model — identifiers, invariants, reproducibility — incremental work can never reach it; when it lives anywhere else, incremental work is cheaper and safer. Decide with the checklist, then encode the new rules so tightly that the compiler and the tests enforce them.