39. Design File Storage and Sync (Google Drive / Dropbox)
1
The interview prompt2
39.1 What do you ask first?3
39.2 How big is this, really?4
39.3 What are the doors?5
39.4 What do we store?6
39.5 What does the whole system look like?7
39.6 How does chunking work?8
39.7 How does de-duplication work?9
39.8 How do sync and conflicts work?10
39.9 How is the metadata database designed?11
39.10 How do presigned uploads work?12
39.11 What would you say in the last five minutes?13
Score yourself14
Variations to practise15
Sources16
Code lab39. Design File Storage and Sync (Google Drive / Dropbox)
39.2 How big is this, really?
3 min read+10 XP
Step 2: Estimates (5 minutes)
Assume 4 MB blocks, 1,000 files per user on average, about 500 bytes of metadata per file, and 2 changed files of 1 MB per user a day.
| What | Working | Result |
|---|---|---|
| Total stored (before de-duplication) | 100 million × 20 GB | ≈ 2 EB |
| New data a day | 100 million × 2 × 1 MB | ≈ 200 TB a day ≈ 2.3 GB/s |
| Blocks in a 1 GB video | 1 GB ÷ 4 MB | 250 blocks |
| File metadata rows | 100 million × 1,000 | 100 billion |
| Metadata size | 100 billion × 500 bytes | ≈ 50 TB |
What the numbers say, out loud:
- "Exabytes of bytes belong in object storage (Chapter 11), deduplicated by content. Never in a database."
- "100 billion metadata rows is big but structured: a sharded database, sharded by user (or by shared folder)."
- "2.3 GB/s of uploads must go straight from clients to storage (presigned URLs), not through our app servers."
Interviewer pushback:
- "How much does de-duplication save?" → It depends on the users: shared team files, the same installer or PDF in thousands of accounts. It's an assumption to measure, but every duplicate block is stored once, and unchanged blocks of edited files are never re-uploaded.
Memory code: "Exabytes in blocks, billions of rows in metadata, bytes never through our servers."