Art of Codeartofcode
LLeeaarrnniinngg
BBlloogg
TToooollss
GGaammeess AArreennaa
JJoobbss
LLiivvee CCoouurrsseess

Courses

  • System DesignHigh-level system design, one everyday scene at a time: 30 concepts and 15 interview rounds.712 lessons
All coursesTutorialsMy Learning

Categories

  • 01AI & Tools
  • 02Tutorials
  • 03Engineering
  • 04Reviews
  • 05Build Logs
  • 06Tales of Misfits
  • 07Opinion
  • 08Career
  • 09News

Read & write

All storiesEvery post, newest first.Write for usPublish your stories and earn from what you write.Writer leaderboardsWho is writing the most read posts.

Categories

  • PDF Tools25
  • Development09
  • Utilities10
  • Design04
  • Converters04
  • Generators02

Featured

JSON FormatterFormat, validate and minify JSON.Regex TesterTest patterns against live text.Merge PDFCombine PDFs in your browser.JWT DecoderRead a token’s header and claims.Password GeneratorStrong passwords, generated locally.Gradient GeneratorBuild CSS gradients visually.
All 53 tools

Play a round

DevdleOne developer word a day, six guesses.Python RushClassic snake, with coding icons to eat.Blink or BreakA quick test of your reflexes.
Games Arena

Learn with us

Live CoursesInstructor-led courses. Join a waitlist and be first in when one starts.LearningStructured programming courses with AI assistance and certificates.

System Design

11. Object / Blob Storage

Lesson 14 of 16

11. Object / Blob Storage

1
The picture to keep in your head
2
11.1 What's the difference between block, file and object storage?
3
11.2 Why not just store files in the database?
4
11.3 What are buckets, keys and metadata?
5
11.4 How do uploads work with presigned URLs?
6
11.5 How do you upload very large files reliably?
7
11.6 How does object storage avoid losing files?
8
11.7 What are storage classes and lifecycle rules?
9
11.8 How should files be served to users?
10
11.9 What happens after a file is uploaded?
11
Memory card and where the analogy breaks
12
Practice: five levels
13
Interview questions
14
Solutions
15
Sources
16
Code lab
11. Object / Blob Storage

Solutions

3 min read+10 XP

Solutions

Level 1

reveal

Level 2

reveal

Level 3

reveal

Level 4

reveal

Level 5 (sketch)

reveal

No tracks available

Lofi Player

artofcode

artofcode.in

The den of devs.

Stories, tools, games, and gear for developers.

Read the StoriesEExxpplloorree TToooollss
Art of Codeartofcode

Platform

  • BBlloogg
  • TToooollss
  • GGaammeess AArreennaa
  • CCoouurrsseess
  • LLeeaarrnniinngg
  • CCaammppuuss

Company

  • AAbboouutt UUss
  • JJoobbss && IInntteerrnnsshhiippss
  • CCoonnttaacctt

Legal

  • TTeerrmmss ooff SSeerrvviiccee
  • PPrriivvaaccyy PPoolliiccyy
  • RReeffuunndd PPoolliiccyy

Copyright © 2026 artofcode.in, All Rights Reserved.

IST·--:--

1Show the Level 1 solution
2**A.**
31. **Database.** Small, structured, queried.
42. **Object storage** for the image; the **key** in the database.
53. **Object storage**, uploaded with multipart; key and metadata in the database.
64. **Database.** Small facts about the files (who, when, which key).
75. **Object storage**, with a lifecycle rule moving older backups to archive and deleting after a year.
8
9**B.**
101. **False.** There are no real folders; "renaming" means copying every object to a new key and deleting the old ones.
112. **False.** Durability protects against hardware loss, not against your own code deleting or overwriting data. Use versioning and backups.
123. **False.** Anyone with the link can use it for 30 days. Use short expiries (minutes) and generate links on demand after checking the purchase.
Show the Level 2 solution
1. 1,000,000 × (500 + 100 + 20) KB = **620 GB**. 3× replication: **1,860 GB**. 10 + 4 erasure coding (1.4×): **868 GB**.
2. 5 GB × 8 = 40,000 megabits ÷ 20 Mbps = 2,000 s ≈ **33 minutes**. 5 GB ÷ 100 MB = **50 parts**. Without multipart, a drop at 90% means resending **all 5 GB**; with multipart, only the part in progress, about **100 MB** (at most a few parts if several were in flight).
3. 50,000,000 × 100 KB = **5 TB a day**. Average: 5 × 10¹² × 8 ÷ 86,400 ≈ **0.46 Gbps** (higher at peak). A CDN serves almost all of it.
1Show the Level 3 solution
2Tested solution (Python 3), the key parts:
3
4```python
5SECRET = b"server-side-secret-key"
6
7
8def _sign(message):
9    return hmac.new(SECRET, message.encode(), hashlib.sha256).hexdigest()
10
11
12def presign(method, key, expires_at):
13    sig = _sign(f"{method}\n{key}\n{expires_at}")          # sign method, key AND expiry
14    return f"https://storage.example.com/{key}?" + urlencode({"expires": expires_at, "sig": sig})
15
16
17def verify(method, url, now):
18    parts = urlparse(url)
19    key = parts.path.lstrip("/")
20    q = {k: v[0] for k, v in parse_qs(parts.query).items()}
21    if not hmac.compare_digest(_sign(f"{method}\n{key}\n{q['expires']}"), q["sig"]):
22        return False
23    return now < int(q["expires"])
24
25
26class ObjectStore:
27    ...
28    def complete_multipart(self, upload_id, content_type="application/octet-stream"):
29        up = self.uploads.pop(upload_id)
30        data = b"".join(up["parts"][n] for n in sorted(up["parts"]))   # join by part number
31        return self.put(up["key"], data, content_type)
32```
33
34All checks pass. The full file, with `put`, `get`, `head`, `list` and the multipart methods, is in `code/ch11_blob_storage.py`.
1Show the Level 4 solution
2**The bug:** the signature covers the **method and key**, but **not the expiry**. The expiry is just a plain number in the URL that `verify` trusts.
3
41. A buyer gets a valid link with `expires=1000` (10 minutes from now).
52. They edit the URL to `expires=99999999999` (thousands of years away).
63. `verify` recomputes the signature from method and key only. It matches, because the expiry was never part of it.
74. `now < 99999999999` is true, so the link works "forever", and they post it on a forum.
8
9**The fix:** include the expiry (and everything else that matters) in the signed message:
10
11```python
12sig = _sign(f"{method}\n{key}\n{expires_at}")
13...
14if not hmac.compare_digest(_sign(f"{method}\n{key}\n{q['expires']}"), q["sig"]):
15    return False
16```
17
18Now changing `expires` changes the expected signature, and the edited link fails. (Real object stores sign the method, path, expiry and relevant headers this way; use their SDKs rather than rolling your own.)
19
203. **More controls:** short expiries generated per download after checking the purchase; **rate-limit** link generation per user; log downloads and alert on unusual volume per purchase; rotate signing credentials if a leak is suspected.
1Show the Level 5 (sketch) solution
21. **Buckets:** `bookstore-public` (covers, behind a CDN), `bookstore-private` (e-books, review originals, videos), `bookstore-backups`. **Keys:** `covers/{book_id}/{size}-v{n}.jpg`, `ebooks/{book_id}/{version}.pdf`, `reviews/{user_id}/{review_id}/original.jpg`, `videos/{video_id}/original.mp4`.
32. **Review photos:** API checks the user, returns a presigned PUT (15 MB size limit, `image/*` only, 10 minutes); the phone uploads directly; a storage event triggers processing. **Videos:** presigned **multipart** upload with 100 MB parts; the app keeps the upload ID to resume; completion triggers transcoding (Chapter 40).
43. **Paid PDFs:** private bucket; the API checks the purchase, then returns a presigned GET valid for a few minutes; rate-limit per user; log downloads.
54. **Covers:** the original goes to `originals/`; an event, a queue and resize workers write three sizes under `covers/` with a version in the key; served through a CDN with long cache times; changing a cover means a new version key.
65. **Lifecycle:** delete unfinished multipart uploads after 7 days; move review originals to Infrequent Access after 90 days; move backups to Archive after 30 days and delete after 1 year. **Protection:** enable **versioning** on the private bucket, restrict delete permissions to a few roles, and copy e-book masters to a second region.