How does block-level delta sync save bandwidth?

A file is split into fixed-size blocks — Dropbox uses 4 MB, hashed with SHA-256 — and each block is named by its content hash. On a save, the client compares the new blocklist to the old one and uploads only the blocks whose hash changed; unchanged blocks are already in storage and are skipped. Editing one block of a ten-block file ships one block, not ten. The idea is the rsync algorithm (Tridgell, 1996): the sender only transmits "literal data for those sections which did not match any of the blocks" the receiver already has.

Why does file-sync metadata need strong consistency instead of eventual?

Because the metadata names which version of a file is current, and if two devices disagree about that, the user sees a file change on one screen but not another — which reads as data loss, the cardinal sin of a filesystem. The bulky blocks can tolerate eventual, loosely-consistent storage (a block is immutable and content-addressed, so a lagging replica still returns identical bytes), but the tiny index cannot. Dropbox rebuilt its sync engine for exactly this: "strong consistency… the server and client have the same view of the remote filesystem before considering a mutation."

Why use HTTP long polling instead of WebSockets for file-change notifications?

The workloads differ in shape. Chat is frequent and bidirectional, so a persistent WebSocket earns its keep. File-change notification is infrequent (folders sit quiet for minutes), one-directional (the server just nudges the client to come pull), and must survive every corporate proxy and firewall. HTTP long polling — the client holds one request open with a cursor, the server answers only when a change lands or a timeout fires — does that with no special infrastructure, degrades to periodic polling, and lets one server hold huge numbers of cheap idle connections. The socket’s duplex streaming is capability this workload never uses.

What happens when two devices edit the same file offline?

The service cannot merge arbitrary binary files, and silently discarding either edit is data loss — so the correct move is first-writer-wins for the name, but keep both files. Whichever save lands first stays as report.txt; the second is preserved beside it as "report (conflicted copy).txt", and a human decides. The mechanism is an optimistic version check: each save carries the base version it branched from, and if the server has already advanced, the compare-and-set fails and the loser becomes a conflicted copy instead of clobbering the winner.

How does a file-sync service keep storage cheap across versions and users?

Content-addressing. Because every block is stored under its own hash, identical blocks are stored once — account-wide and often globally, so the shared installer or forwarded attachment costs one physical copy. Version history is nearly free too: a version is just a blocklist (a small ordered set of hashes), so keeping history means keeping old lists while the expensive bytes stay shared. Three levers bound the growth — version caps with garbage collection of unreferenced blocks, cold storage for untouched blocks, and reference counting so a block is deleted only when the last file pointing at it is gone.

Design a File-Sync Service

Drop a file in a folder and it appears, seconds later, on every device you own. Build the Dropbox shape from zero: cut files into content-hashed 4MB blocks so editing one block ships one block, guard a tiny metadata index with strong consistency while the bulky bytes stay loose, nudge devices with a held long-poll connection instead of a poll storm, and reconcile offline edits into first-writer-wins conflicts that never silently merge — with the delta-sync bandwidth save computed live.

System design · Systems. The source ↗

A free, interactive, animated visual explainer of Design a File-Sync Service — built to be understood, not skimmed.

Questions

How does block-level delta sync save bandwidth?: A file is split into fixed-size blocks — Dropbox uses 4 MB, hashed with SHA-256 — and each block is named by its content hash. On a save, the client compares the new blocklist to the old one and uploads only the blocks whose hash changed; unchanged blocks are already in storage and are skipped. Editing one block of a ten-block file ships one block, not ten. The idea is the rsync algorithm (Tridgell, 1996): the sender only transmits "literal data for those sections which did not match any of the blocks" the receiver already has.
Why does file-sync metadata need strong consistency instead of eventual?: Because the metadata names which version of a file is current, and if two devices disagree about that, the user sees a file change on one screen but not another — which reads as data loss, the cardinal sin of a filesystem. The bulky blocks can tolerate eventual, loosely-consistent storage (a block is immutable and content-addressed, so a lagging replica still returns identical bytes), but the tiny index cannot. Dropbox rebuilt its sync engine for exactly this: "strong consistency… the server and client have the same view of the remote filesystem before considering a mutation."
Why use HTTP long polling instead of WebSockets for file-change notifications?: The workloads differ in shape. Chat is frequent and bidirectional, so a persistent WebSocket earns its keep. File-change notification is infrequent (folders sit quiet for minutes), one-directional (the server just nudges the client to come pull), and must survive every corporate proxy and firewall. HTTP long polling — the client holds one request open with a cursor, the server answers only when a change lands or a timeout fires — does that with no special infrastructure, degrades to periodic polling, and lets one server hold huge numbers of cheap idle connections. The socket’s duplex streaming is capability this workload never uses.
What happens when two devices edit the same file offline?: The service cannot merge arbitrary binary files, and silently discarding either edit is data loss — so the correct move is first-writer-wins for the name, but keep both files. Whichever save lands first stays as report.txt; the second is preserved beside it as "report (conflicted copy).txt", and a human decides. The mechanism is an optimistic version check: each save carries the base version it branched from, and if the server has already advanced, the compare-and-set fails and the loser becomes a conflicted copy instead of clobbering the winner.
How does a file-sync service keep storage cheap across versions and users?: Content-addressing. Because every block is stored under its own hash, identical blocks are stored once — account-wide and often globally, so the shared installer or forwarded attachment costs one physical copy. Version history is nearly free too: a version is just a blocklist (a small ordered set of hashes), so keeping history means keeping old lists while the expensive bytes stay shared. Three levers bound the growth — version caps with garbage collection of unreferenced blocks, cold storage for untouched blocks, and reference counting so a block is deleted only when the last file pointing at it is gone.

Design a File-Sync Service

Questions

Related explainers