Art of Codeartofcode
Lesson 2 of 16

11. Object / Blob Storage

1
The picture to keep in your head
2
11.1 What's the difference between block, file and object storage?
3
11.2 Why not just store files in the database?
4
11.3 What are buckets, keys and metadata?
5
11.4 How do uploads work with presigned URLs?
6
11.5 How do you upload very large files reliably?
7
11.6 How does object storage avoid losing files?
8
11.7 What are storage classes and lifecycle rules?
9
11.8 How should files be served to users?
10
11.9 What happens after a file is uploaded?
11
Memory card and where the analogy breaks
12
Practice: five levels
13
Interview questions
14
Solutions
15
Sources
16
Code lab
11. Object / Blob Storage

11.1 What's the difference between block, file and object storage?

3 min read+10 XP

At the cloakroom: There are three ways to keep things. A locker you rent that's yours alone, where you can rearrange things inside however you like. A shared office cupboard with labelled shelves and folders that a team uses together. And the cloakroom: hand over a whole item, get a token, collect it whole.

In the system:

block, file and object storage compared.

Figure: block storage is a raw disk, file storage is a shared folder tree, object storage is key to whole object.

  • Block storage is a raw disk made of numbered blocks, attached to one server at a time (a cloud volume like AWS EBS, or the SSD in your laptop). A filesystem or a database sits on top. Fast for small, random reads and writes, which is why databases live on it.
  • File storage is a shared filesystem: folders, files, rename, append, shared by several servers over the network (NFS, AWS EFS). Familiar, but hard to grow to billions of files and huge traffic.
  • Object storage stores whole objects under a key in a bucket, with metadata, accessed over HTTP. There are no real folders: the namespace is flat, and covers/42/large.jpg is just a key that happens to contain slashes. Objects are written and replaced whole. It scales to practically unlimited objects and size, cheaply, and any server or client can reach it. Examples: Amazon S3, Google Cloud Storage, Azure Blob Storage, Cloudflare R2, and self-hosted MinIO. Many of these speak S3's API.

Memory code: "Locker, cupboard, cloakroom." Block for databases, file for shared folders, object for everything big and whole.