Art of Codeartofcode
Lesson 3 of 16

12. Caching: The Basics

1
The picture to keep in your head
2
12.1 What is a cache, and what are a hit, a miss and the hit rate?
3
12.2 Why is a cache faster than the database?
4
12.3 Where does a cache physically live?
5
12.4 On a miss, who fetches the data?
6
12.5 When the data changes, what gets written where?
7
12.6 What happens when the real data changes, and why add jitter?
8
12.7 What happens when the cache is full?
9
12.8 How do you invalidate safely, and what's the race everyone hits?
10
12.9 When should you cache, what should you never cache, and how do you name keys?
11
Memory card and where the analogy breaks
12
Practice: five levels
13
Interview questions
14
Solutions
15
Sources
16
Code lab
12. Caching: The Basics

12.2 Why is a cache faster than the database?

3 min read+10 XP

Why is asking the database slow?

In the kitchen: A supermarket trip isn't one step. You travel there, the shopkeeper figures out which aisle, walks to the shelf (sometimes the back storeroom), and you carry it home.

In the system: For every query, the database usually has to:

  1. Receive the query over the network.
  2. Plan how to answer it (parse the query, choose an approach).
  3. Find the rows, sometimes reading from disk (the back storeroom).
  4. Send the result back over the network.

Every one of those steps costs time, for every request, even if the question was asked a second ago.

Memory code: "Receive, Plan, Find, Send" = Really Patient Folks Shop. The database is a patient shopkeeper. Patient means slow.

Why is a cache faster?

In the kitchen: The fridge is close (in your kitchen, not across town) and simple (you open the door and grab the box with the label "milk". Nobody has to figure anything out).

In the system: Same two reasons.

  • Close: a cache keeps its data in RAM (memory). Reading from RAM takes roughly 100 nanoseconds. One network round trip inside a data centre takes roughly half a millisecond, about 5,000 times longer. (Approximate figures from Jeff Dean's "latency numbers every programmer should know"; Chapter 2, section 2.4.)
  • Simple: a cache doesn't plan queries, join tables or filter rows. You give it an exact key, it hands back the value.

if reading RAM took 1 second, a trip to Redis would take about 83 minutes and a database query hours.

Figure: scale RAM up to 1 second and a network round trip becomes about 83 minutes.

"But isn't Redis on another server? That's a network trip too." Yes. Redis is like a shared pantry down the hall: a short walk, but still far quicker than the supermarket, because everything's already on the shelf and nobody has to look for it. A cache inside your app's own memory is like your pocket: no walk at all.

Memory code: "Close and Simple." The fridge wins because it's close (RAM) and simple (exact key, no searching).

If a cache is so fast, why not keep everything in it?

In the kitchen: Why not move the whole supermarket into your fridge? Because the fridge is little (it holds a few things, not a warehouse), forgetful (unplug it and things go bad) and clueless (you can't ask it "show me everything under ₹100 from last week". It just holds labelled boxes).

In the system:

  1. Little: RAM is limited and expensive. A database holds terabytes on disk; memory costs far more per gigabyte.
  2. Forgetful: most caches lose their data on restart. Databases are built to survive crashes.
  3. Clueless: a cache only answers "give me the value for this exact key". It can't run queries like "all orders over ₹500 from last week".

So the cache holds the small set of data asked for the most. The database holds everything.

Memory code: the fridge is Little, Forgetful and Clueless.

Why not just make the database faster instead?

In the kitchen: You could hire a faster shopkeeper. But if 1,000 people want the same milk, even the fastest shopkeeper fetches it 1,000 times. Keeping it in the fridge means it gets fetched once.

In the system: You should make the database faster too (indexes, better queries, read replicas; Chapters 9 and 14). But a faster database still does the work for every request. A cache skips the work for repeated requests. When thousands of users ask the identical question, not doing the work beats doing it faster.

Memory code: "Don't cook faster. Cook once."