39. Design File Storage and Sync (Google Drive / Dropbox)
1
The interview prompt2
39.1 What do you ask first?3
39.2 How big is this, really?4
39.3 What are the doors?5
39.4 What do we store?6
39.5 What does the whole system look like?7
39.6 How does chunking work?8
39.7 How does de-duplication work?9
39.8 How do sync and conflicts work?10
39.9 How is the metadata database designed?11
39.10 How do presigned uploads work?12
39.11 What would you say in the last five minutes?13
Score yourself14
Variations to practise15
Sources16
Code lab39. Design File Storage and Sync (Google Drive / Dropbox)
39.5 What does the whole system look like?
3 min read+10 XP
Step 4: High-level design (10 minutes)
Figure: the final design. Small metadata goes through our services; big bytes go straight to storage.
Uploading a change, step by step:
- The desktop client notices a changed file, splits it into blocks (39.6), and hashes each with SHA-256.
- It sends the list of hashes to the metadata service: "which of these do you not have?"
- The service answers with the missing hashes and a presigned upload URL for each (39.10).
- The client uploads only those blocks directly to the block store.
- It commits the new version (path, base version, list of hashes). The metadata service checks the base version (39.8), writes the new version and a journal entry in one transaction.
- The notifier tells the user's other devices (and people sharing the folder) that something changed.
Syncing on another device:
- The phone gets the notification (or its long-poll returns).
- It asks for changes since its cursor, gets the new version's block list, downloads only the blocks it doesn't have locally, and rebuilds the file.
Interviewer pushback:
- "What if the upload is interrupted halfway through a 50 GB video?" → Blocks already uploaded are stored; on resume, the "which are missing?" check skips them. The commit happens only when all blocks are present, so nobody sees a half file.
- "How do you garbage-collect blocks?" → A block can be deleted only when no file version references it: keep reference counts (or periodic mark-and-sweep) and delete blocks after versions age out of the 30-day history.
Memory code: "Hash the blocks, upload the missing ones, commit the list, tell the other devices."
Offer: "The interesting parts are chunking, de-duplication, sync conflicts, the metadata database, and uploads. Which would you like?"