39. Design File Storage and Sync (Google Drive / Dropbox)
1
The interview prompt2
39.1 What do you ask first?3
39.2 How big is this, really?4
39.3 What are the doors?5
39.4 What do we store?6
39.5 What does the whole system look like?7
39.6 How does chunking work?8
39.7 How does de-duplication work?9
39.8 How do sync and conflicts work?10
39.9 How is the metadata database designed?11
39.10 How do presigned uploads work?12
39.11 What would you say in the last five minutes?13
Score yourself14
Variations to practise15
Sources16
Code lab39. Design File Storage and Sync (Google Drive / Dropbox)
39.10 How do presigned uploads work?
3 min read+10 XP
Clients upload directly to object storage, so 2.3 GB/s never touches our app servers (Chapter 11):
- After "which blocks are missing?", the metadata service returns a presigned URL for each: the block's hash, an expiry, and an HMAC signature over both made with a secret only our service knows.
- The storage layer checks the signature and expiry before accepting the upload, and checks that the uploaded bytes really hash to the name in the URL. In the chapter's code, a valid URL works before its expiry, fails after it, and fails if someone swaps the block hash.
- Big blocks or files use multipart uploads (on S3: parts from 5 MiB to 5 GiB, up to 10,000 parts), which also makes retries cheap.
- Downloads work the same way: short-lived signed URLs, often through a CDN, issued only after an access check (owner or shared with this user).
Memory code: "We sign the permission slip; the bytes go straight to storage."