Design Dropbox: File Storage and Sync

Understanding the problem

This is the design Dropbox is most likely to hand you, and it is the one they built the company on. Read the prompt carefully, because the interesting word is "sync," not "store." Storing a file is easy. Syncing it is not. The same folder lives on many machines at once. Those machines go offline and come back, two of them can edit the same file, and the network in between is too slow to re-upload a whole file on every save. A good design keeps every device converging on the same state while sending only the parts that changed.

Get the altitude right before drawing a box. Sync splits cleanly into two problems that want different machinery. One is bookkeeping: which files exist, what path each sits at, what version is current, and which chunks make up each file. That data is small, changes transactionally, and has to be correct. The other is the bytes themselves: gigabytes to petabytes of file content that is written once and read many times. Keep those two planes apart in your head from the first sentence, because conflating them is the single most common way this round goes wrong.

The Dropbox angle

This is the flagship Dropbox design, and the thing that makes it a Dropbox round rather than a generic "design Google Drive" is block-level sync. Anyone can draw a client uploading files to blob storage. A Dropbox answer treats a file as a list of content-addressed chunks. Editing a 4 GB video re-sends only the few hundred kilobytes that changed, and identical chunks are stored exactly once across all files and all users. Dropbox is openly public about this architecture, so the interviewer has usually worked on the real thing. Lead with chunks, hashing, and dedup, and the design reads as Dropbox instead of a whiteboard sketch of a file server.

Functional requirements

Scope it to the sync loop and let the interviewer pull you outward. The core is one folder staying identical across devices while the client and server exchange the minimum amount of data.

Non-functional requirements

Reason through the targets, because they are what force the block-level design. The defining property is bandwidth economy. Users keep large files in their folders and edit them often, so the data on the wire has to track the size of the edit instead of the size of the file. Saving a one-line change to a large document should cost a few kilobytes. That single requirement is what rules out treating a file as an opaque blob you re-upload.

Consistency splits by plane. Metadata is strongly consistent. A device must never see a file at a path that was already moved, and a commit that lists a file's chunks lands all-or-nothing, so no reader ever reconstructs a half-written file. Cross-device propagation, on the other hand, is allowed to be eventually consistent within a few seconds, because a change physically cannot appear on another continent instantly and users accept a short lag. Durability is close to absolute once bytes are acknowledged: a file the user saw sync must not be lost. And the client has to keep working entirely offline, queueing edits locally and reconciling them the moment the network returns.

• Bandwidth: Track the edit, not the file. Editing a large file must send bytes proportional to what changed, not the whole file. This is the requirement that forces chunking plus content addressing: split the file into blocks, hash each block, and send only the blocks whose hash is new.
• Metadata consistency: Strong, atomic commits. A commit that records a file's path, version, and chunk list applies whole or not at all. Readers see the old file or the new one, never a partial chunk list, so no device reconstructs a corrupt file.
• Propagation: Seconds cross-device. A change on one device should surface on the others within a few seconds. This can be eventually consistent: near-real-time is the target, instantaneous is not required, which is what lets a notification-plus-pull model work.
• Offline + durability: Work offline, never lose an ack. The client queues edits locally while offline and reconciles on reconnect. Once the server acknowledges a chunk and a commit, that data is durable and must survive machine failures.

Back-of-the-envelope: what has to scale

Put rough numbers down to find the hard part. Start from users. Take on the order of 500 million registered users, roughly 100 million active on a given day, about 3 devices each, and something like 200 files per user on average. That is around 100 billion files in the system. At an average file size near 100 KB, total stored content is about 100 billion times 100 KB, which lands around 10 PB before you count version history and replication. So the byte plane is petabyte-scale and the file plane is hundred-billion-scale. The two are wildly different in size per object. A file's metadata row is a few hundred bytes while its content averages 100 KB and can run to gigabytes.

Now the chunks, because they are what move. Blocks are fixed at up to 4 MiB, so a typical small file is a single block while a 4 GB video is about a thousand blocks. Content addressing means identical blocks are stored once. Cross-file and cross-user duplication (the same PDF in a hundred inboxes, an unchanged block in a re-saved file) collapses real storage well below the raw 10 PB. The payoff shows up on edits. Saving a one-line change to that 4 GB video rehashes its blocks, finds one block changed, and uploads roughly 4 MiB instead of 4 GB. That is a thousand-fold saving on the wire. The change rate is the load that never sleeps: across 100 million daily-active devices, small edits arrive constantly, and each one is a cheap metadata commit plus at most a handful of new blocks.

Two more numbers set the real-time side. Propagation latency has a budget of a few seconds end to end, so the notification path has to be push-like rather than a slow poll. Connection count is large too. On the order of a million devices hold an open channel at any minute, waiting to be told about changes. The mechanism that keeps those connections cheap is itself a design decision. Hold this shape going in: a small, strongly-consistent, high-commit-rate metadata plane, a huge immutable deduplicated byte plane, and a low-latency notification path fanning changes to millions of waiting devices.

The set-up

Core entities

Name the objects before you draw. Four carry this design, and the relationship between a file and its blocks is the spine everything else hangs off.

One relationship does most of the work. A File points at an ordered list of Block hashes, and the blocks are immutable content. Change the data and you do not mutate anything: you write the new blocks, and record a new file version whose block list references the new blocks plus all the unchanged ones. Keep the old version's block list and you can still reconstruct the file as it was. That indirection, file to block-list to immutable blocks, is what makes edits cheap and gives you dedup and version history. It is why sync comes down to moving a short list of hashes plus whatever blocks are genuinely new.

API

The client talks to two surfaces that mirror the two planes. Metadata calls commit and read the block lists and the change log; block calls move the actual bytes and are keyed by content hash so they dedup and retry safely. The call worth staring at is has_blocks: before uploading anything, the client asks which of its block hashes the server is missing, and uploads only those. That one round trip is where the bandwidth savings live.

High-level design

Build it around the two planes. On the client sits a sync engine that watches the folder, chunks and hashes changed files, and journals what it did. On the server, metadata calls flow through an API gateway to a metadata service backed by a strongly-consistent store, while block calls move bytes to a block service backed by immutable blob storage. A notification service tells other devices when a namespace changed. The diagram codes each box by role: blue for the devices, violet for the sync and server services, amber for the journal and the long-poll channel, green for the durable stores.

1) Detect and chunk a local change. The File watcher hears the operating system fire a filesystem event the moment a file is saved, and hands it to the Sync client. The client reads the changed file and splits it into fixed blocks of up to 4 MiB, hashing each block as it goes. For a re-saved file, most of those hashes match the file's previous version, so the client can tell precisely which blocks are new: usually just the one or two the edit touched. It records the intended operation in the Local journal first, so a crash mid-sync leaves a clear record of what still needs to happen. The trap here is chunking strategy, because fixed blocks handle overwrites well but stumble on inserts, which the block-diffing deep dive takes apart.

2) Upload only the new bytes. Before sending anything, the Sync client calls has_blocks with the file's block hashes, and the Block service answers which ones it is missing. The client uploads exactly those blocks with put_block. Each block is named by its content hash, so the upload is idempotent: a retry after a dropped connection re-puts the same hash and is a harmless no-op. Editing that one-line change in a 4 GB file sends roughly 4 MiB, not 4 GB. The bytes go to the block service and land in Block storage, and they deliberately never pass through the metadata path. In production the client is handed a presigned URL and streams blocks straight to blob storage, so no app server sits in the byte path at all.

3) Commit the metadata atomically. With the new blocks stored, the Sync client calls commit_batch through the API gateway with its operations and the cursor it was working from. The Metadata service applies the batch to the Metadata DB as one atomic transaction: it writes the new file version and block list, advances the namespace's operation log, and moves the cursor forward, all-or-nothing. If the device's base cursor is stale, meaning another device committed in the meantime, the service rejects the commit and the client re-syncs first, which is where conflict handling begins. Bytes are already durable at this point, so the commit is small and fast even though it is the strict part.

4) Tell the other devices, and let them pull. A successful commit makes the Metadata service publish an event to the Notification service, which signals every one of the user's Other devices that is waiting on the Long-poll channel for that namespace. A signaled device wakes, calls get_delta with its cursor, and gets back the compact list of what changed since it last synced. It then fetches only the blocks it does not already have and reconstructs the files locally. A device that was offline for a week does the same thing on reconnect, just with a longer delta. The long-poll versus poll tradeoff, and how reconnecting devices avoid a thundering herd, is the change-notification deep dive.

Technology choices

Every box is a category until you name a real system and defend it under failure. The two that decide this round are what stores the blocks and what stores the metadata, because the whole design rests on keeping them apart.

• Block storage is an immutable content-addressed blob store, which at scale means either a cloud object store like Amazon S3 or a purpose-built system (Dropbox runs its own, Magic Pocket, after migrating off S3). You reach for object storage because the bytes are huge write-once files that a database would buckle under. Content addressing (naming each block by its own hash) gives you automatic dedup, since identical content collides to one key. Writes become idempotent, since re-uploading a block just overwrites it with itself. Sharing is safe, since an immutable block can be referenced by many files and users without anyone mutating it underneath. The failure probe is durability, and the answer is that a dedicated store replicates or erasure-codes each block across zones and verifies it in the background, which the block-storage lesson goes into.
• Metadata DB is a strongly-consistent, horizontally-sharded transactional store, sharded by user or namespace so one user's files and operation log sit together. A relational database like MySQL fronted by a sharding and caching layer is the recognizable pick (Dropbox built Edgestore over sharded MySQL, and a transactional key-value direction on top). You need transactions because a commit that writes a version, a block list, and a cursor must be atomic. You shard by namespace because sync is almost always scoped to one user's tree, so that keeps a commit and a delta on a single shard. The probe is a hot shard from a heavy user, and the answer is that namespace sharding plus read replicas for the delta reads keeps any one commit stream local.
• The block service and the API gateway are stateless application servers behind a load balancer, split so the byte path and the metadata path scale independently. You separate them because they have opposite shapes. The block path is high-throughput bytes, best served by presigned direct-to-blob uploads that skip the app tier entirely. The metadata path is many tiny latency-sensitive transactions. Because neither holds durable state, an instance dying mid-request just drops from the pool and the client retries, with an in-flight metadata transaction rolling back cleanly in the store. The probe is what happens when a block upload fails halfway, and the answer is that content-hash naming makes the retry idempotent, so the client simply re-puts the missing blocks.
• The notification service over the Long-poll channel is a publish-subscribe fan-out (a broker such as Kafka or a Redis pub-sub layer feeding per-device long-poll or websocket connections). You use pub-sub because a commit on one device has to reach the user's other devices within seconds, without those devices polling constantly. Long-poll is the cheap default. An idle blocked request costs almost nothing while a busy poll every few seconds across a million devices is a needless flood. The probe is the reconnect storm when a data-center blip drops a million connections at once. The answer is jittered reconnect backoff, and the reconnect itself is just a cursor-based delta, so a device that missed nothing gets an empty response cheaply.

The Dropbox angle

The one instinct a Dropbox interviewer is listening for is separating the metadata plane from the block plane, so make that split explicit and early. Metadata is small, transactional, strongly consistent, and sharded; blocks are huge, immutable, deduplicated, and served straight from blob storage without touching an app server. A candidate who routes file bytes through the metadata service, or who tries to make the blob store transactional, has fused two things Dropbox spent years keeping apart. Say the split in one sentence, then let every later decision follow from it. Cheap edits come from block-level diffing over that byte plane, and correct sync comes from atomic commits over that metadata plane.

Potential deep dives

The deep dives are where the round is decided. A Dropbox interviewer drives at four things that make sync hard. How do you diff a file cheaply? How does the client engine stay correct through crashes and concurrent work? What happens when two devices edit the same file? And how does a device learn about remote changes fast without hammering the server?

1) How do we sync only the bytes that changed?

This is the block-level-diffing question, and it is the heart of the design. The math motivates it: a user editing a 4 GB video should not re-upload 4 GB for a one-line change. If a file is a list of block hashes, the client can rehash the file, compare hashes against the previous version, and find that only one block moved. The question is how to cut a file into blocks so that a small edit changes as few of them as possible.

The Dropbox angle

Name the hash function and the dedup scope, because that specificity is what reads as Dropbox rather than a generic answer. Blocks are identified by a SHA-256-class digest, and dedup is global: the same block written by any file or any user is stored once. The follow-up that separates strong candidates is the privacy caveat, so raise it before you are asked. Naive cross-user dedup by hash can leak whether a file already exists. Upload a block, and a suspiciously fast "already have it" tells you someone else has that exact content. So dedup has to be scoped carefully instead of blindly global across all users. Mentioning that trade-off unprompted shows you have thought past the happy path.

2) How does the client sync engine stay correct?

The client is where sync lives, and it is a harder concurrency problem than the server. It has to watch a folder that the user and the operating system are changing underneath it, and upload and download at the same time. It has to survive being force-quit mid-transfer and never leave the local folder in a state that disagrees with the server. The engine has three parts working together: the watcher, the journal, and a single control loop.

The loop that ties them together is straightforward once the parts are named. The watcher produces local changes; the control thread turns each into a journaled operation, chunks and hashes the file, uploads the new blocks, and commits the metadata. In the other direction, a notification triggers a delta pull, and the control thread applies each remote operation: fetch missing blocks, write the reconstructed file, advance the cursor. Both directions funnel through the one control thread, so an upload and a download of the same file can never race, and the journal means any interruption resumes cleanly from the last durable step.

The Dropbox angle

Dropbox's coding rounds lean hard on concurrency, and this design is where that shows up in system design, so make the single-writer control loop an explicit choice. The naive instinct is to spin up threads for uploads, downloads, and the watcher, then guard shared state with locks. Dropbox moved away from exactly that design because it bred subtle races and deadlocks. Say instead that the sync decisions run on one control thread with futures. Only the blocking work (network, hashing, disk) is offloaded, so the state machine itself is never concurrently mutated. Pair that with the journal-before-action rule and you have described the two properties that keep a real sync engine correct: single-writer scheduling and durable intent.

3) What happens when two devices edit the same file?

Concurrent edits are unavoidable: a laptop goes offline on a plane, edits a file, and meanwhile the phone edits the same file and syncs. When the laptop reconnects, both have a new version built from the same starting point, and the system has to resolve it without silently throwing an edit away. The cursor check in commit_batch surfaces the conflict. The laptop's commit is built on a base cursor the server has already moved past, so the commit is rejected and the client must reconcile before retrying.

The Dropbox angle

The phrase the interviewer wants to hear is "conflicted copy," and the boundary they want you to draw is against collaborative editing. File sync keeps both versions because it cannot understand the bytes and refuses to lose an edit; a collaborative editor merges character operations because it can. A strong answer states Dropbox's conflicted-copy behavior and explains that last-writer-wins is unacceptable because it destroys data silently. Then it heads off the obvious follow-up by noting that operational transform and CRDTs solve a genuinely different problem: concurrent edits to one structured document, not reconciliation of two opaque file versions. Reaching for auto-merge on arbitrary files is the tell that the two problems have blurred together.

4) How does a device learn about remote changes?

A change committed on the laptop has to reach the phone within seconds, and the phone cannot know to ask unless something tells it. With a million or more devices idle at any moment, the mechanism that carries that signal has to be near-real-time and nearly free per idle device. The cursor makes this tractable. Each device holds an opaque marker of how far it has synced in a namespace. Catching up is always "give me the changes since this cursor," whether the gap is one edit or a week of them.

The Dropbox angle

Dropbox exposes this as a long-poll endpoint, so the cursor-plus-long-poll model maps straight onto the real system. Separate the notification from the payload. The long-poll carries only a "something changed" signal. The device then pulls the delta by cursor and fetches only missing blocks, so the connection stays cheap and the heavy work happens on demand. Fold in the offline-catch-up case for free, since a device returning after a week hits the same get_delta path with an older cursor, and it just gets a longer change list. If the interviewer pushes on scale, the answer is the reconnect-storm backoff and the per-connection memory budget, not a heavier protocol.

What is expected at each level

That is a lot of surface. How much you need to land depends on the level the loop is calibrated for. Dropbox settles level against its public engineering framework, and the design round is one of the rounds that reads seniority, so tune the depth to the target.

IC3 / IC4 (Software Engineer / Senior)

Breadth-led, with one real depth. Reach a working end-to-end shape. A client chunks files into content-addressed blocks and uploads only the new ones, and a metadata plane commits block lists atomically. A block plane stores immutable deduplicated chunks, and a notification path gets a change to other devices. The non-negotiable is the plane separation and block-level sync, so justify why editing a big file sends only its changed blocks. Getting the metadata-versus-block split clean and explaining has_blocks plus dedup is a solid IC3, and reaching one deep dive (usually chunking or conflicts) with prompting pushes toward IC4.

IC5 (Staff)

Depth now, and this is the level the round most often decides. You lead with the two-plane model unprompted and drive at least two deep dives on your own. One is the fixed-versus-content-defined chunking trade-off, with the front-insert case named. Another is conflict resolution landing on conflicted copies, with the collaborative-editing boundary drawn. You reason about the client sync engine as a single-writer control loop with a durable journal rather than a pile of locked threads, and you name the real stores and defend them. The bar is a near-complete sync design where you found the hard parts, block diffing, concurrency, and conflicts, without being led to them.

IC6+ (Principal)

Depth leads as the scope widens. Expect the dedup privacy caveat reasoned unprompted, and the notification path sized concretely (a million connections, reconnect-storm backoff, per-connection memory). The metadata store's sharding-by-namespace and hot-shard behavior is argued from the traffic. The offline-catch-up and crash-recovery paths are handled from the journal and cursor model. You reason about how the system degrades and heals when a zone drops or a heavy user goes hot, and you hold the wheel while the interviewer probes from the edges. The bar is that every consistency and storage choice is deliberate and defended, and the block-versus-metadata split is the axis the whole design turns on.