Art of Codeartofcode
Lesson 10 of 15

13. Caching at Scale

1
The picture to keep in your head
2
13.1 What is a cache stampede, and how do you stop one?
3
13.2 What is cache penetration?
4
13.3 What is a cache avalanche?
5
13.4 What is a hot key, and why can't sharding fix it?
6
13.5 How do you spread a cache across many machines?
7
13.6 What is cache warming?
8
13.7 How does HTTP caching work in browsers and CDNs?
9
13.8 How do you know your cache is healthy?
10
Memory card and where the analogy breaks
11
Practice: five levels
12
Interview questions
13
Solutions
14
Sources
15
Code lab
13. Caching at Scale

Memory card and where the analogy breaks

3 min read+10 XP

The memory card (tear this page out)

#QuestionMemory code
13.1Stampede?One person goes to the shop; everyone else waits at the fridge
13.2Penetration?Write "we don't sell that" on the fridge door
13.3Avalanche?Don't let the whole fridge go off, or switch off, at once
13.4Hot keys?One pot can't feed the whole wedding: make more pots
13.5Distributed cache?Split the stock across fridges by label, and keep a spare of each
13.6Warming?Stock the fridge before the guests arrive
13.7HTTP caching?Fresh: don't ask. Stale: ask "still good?"
13.8Measuring?Watch the hits, the throw-outs, the fullness and the shop's queue

The whole chapter in one line: At scale, protect the shop: one person per errand, notes for missing dishes, staggered use-by dates, more pots for the favourite, and a stocked fridge before the doors open.

Where the analogy breaks

  • Wedding guests are patient. Requests time out, retry and add even more load, so a stampede can feed itself (Chapter 29).
  • A bad guest asking for missing dishes is rare. Bots asking for missing data on purpose are common, which makes penetration a security problem, not just a performance one.
  • Food can't be in two fridges at once. Data can, and copying a hot key onto several nodes is exactly the fix.

Quick check

Five questions on this chapter. Pick an answer, then check it.

quiz