Lesson 6 of 16
39. Design File Storage and Sync (Google Drive / Dropbox)

39.5 What does the whole system look like?

3 min read+10 XP

Step 4: High-level design (10 minutes)

the desktop client chunks and hashes files, asks the metadata service which blocks are missing, uploads them directly to the block store with presigned URLs, commits the new version; a notifier tells other devices, which sync from their cursor.

Figure: the final design. Small metadata goes through our services; big bytes go straight to storage.

Uploading a change, step by step:

  1. The desktop client notices a changed file, splits it into blocks (39.6), and hashes each with SHA-256.
  2. It sends the list of hashes to the metadata service: "which of these do you not have?"
  3. The service answers with the missing hashes and a presigned upload URL for each (39.10).
  4. The client uploads only those blocks directly to the block store.
  5. It commits the new version (path, base version, list of hashes). The metadata service checks the base version (39.8), writes the new version and a journal entry in one transaction.
  6. The notifier tells the user's other devices (and people sharing the folder) that something changed.

Syncing on another device:

  1. The phone gets the notification (or its long-poll returns).
  2. It asks for changes since its cursor, gets the new version's block list, downloads only the blocks it doesn't have locally, and rebuilds the file.

Interviewer pushback:

  • "What if the upload is interrupted halfway through a 50 GB video?" → Blocks already uploaded are stored; on resume, the "which are missing?" check skips them. The commit happens only when all blocks are present, so nobody sees a half file.
  • "How do you garbage-collect blocks?" → A block can be deleted only when no file version references it: keep reference counts (or periodic mark-and-sweep) and delete blocks after versions age out of the 30-day history.

Memory code: "Hash the blocks, upload the missing ones, commit the list, tell the other devices."


Offer: "The interesting parts are chunking, de-duplication, sync conflicts, the metadata database, and uploads. Which would you like?"