Art of Codeartofcode
Lesson 12 of 16

11. Object / Blob Storage

1
The picture to keep in your head
2
11.1 What's the difference between block, file and object storage?
3
11.2 Why not just store files in the database?
4
11.3 What are buckets, keys and metadata?
5
11.4 How do uploads work with presigned URLs?
6
11.5 How do you upload very large files reliably?
7
11.6 How does object storage avoid losing files?
8
11.7 What are storage classes and lifecycle rules?
9
11.8 How should files be served to users?
10
11.9 What happens after a file is uploaded?
11
Memory card and where the analogy breaks
12
Practice: five levels
13
Interview questions
14
Solutions
15
Sources
16
Code lab
11. Object / Blob Storage

Practice: five levels

3 min read+10 XP

Assignments

Five levels. Do them in order. Answers are in "Solutions" at the end, so try first.

Level 1: Think (no code)

A. Database, object storage, or both? One line each.

  1. A book's title, price and author.
  2. The book's cover image.
  3. A 300 MB author interview video.
  4. Which user uploaded which review photo, and when.
  5. Daily database backups kept for a year.

B. True or false? Explain.

  1. "Object storage has folders, so I can rename the folder covers/ to images/ instantly."
  2. "Eleven nines of durability means I don't need backups."
  3. "Presigned URLs are safe to give out with a 30-day expiry for paid downloads."

Level 2: Calculate

  1. The bookstore has 1 million books, each with three cover sizes of 500 KB, 100 KB and 20 KB. How much storage? With 3× replication? With 10 + 4 erasure coding?
  2. A 5 GB video is uploaded over a 20 Mbps connection. How long does it take? In 100 MB parts, how many parts? If the connection drops at 90%, how much is re-sent with and without multipart?
  3. Covers get 50 million views a day at 100 KB each. How much data a day leaves storage? What's the average bandwidth in Gbps?

Level 3: Build

In Python (standard library only), build a tiny object store:

  1. put(key, data, content_type, meta), get(key), head(key) (size, type, ETag, your metadata) and list(prefix).
  2. Multipart: start_multipart, upload_part(upload_id, n, data) (any order, retries allowed), complete_multipart.
  3. presign(method, key, expires_at) using an HMAC signature, and verify(method, url, now).

Check that: listing by prefix works; objects are replaced whole; a presigned URL works before expiry and fails after; changing the method, the key, or the expiry breaks it; multipart parts uploaded out of order (with one retried) produce exactly the original bytes.

Level 4: Debug

Customers have been sharing paid e-book download links on forums, and the links keep working for months, though they were meant to expire in 10 minutes. The code:

python
  1. Find the bug. Explain step by step how someone makes a link that works forever.
  2. Fix it.
  3. Name one more control you'd add for paid downloads.

Level 5: Design

Design the bookstore's file handling:

  • Cover images for 1 million books, shown on every page.
  • Paid e-book PDFs (up to 50 MB), downloadable only by buyers.
  • Review photos uploaded from phones (up to 15 MB).
  • Author videos up to 5 GB.

Deliver:

  1. Buckets and key layout.
  2. The upload flow for review photos and for videos.
  3. How paid PDFs are protected.
  4. How covers are processed and served.
  5. Lifecycle rules and one way you'd protect against accidental deletion.