13. Caching at Scale
1
The picture to keep in your head2
13.1 What is a cache stampede, and how do you stop one?3
13.2 What is cache penetration?4
13.3 What is a cache avalanche?5
13.4 What is a hot key, and why can't sharding fix it?6
13.5 How do you spread a cache across many machines?7
13.6 What is cache warming?8
13.7 How does HTTP caching work in browsers and CDNs?9
13.8 How do you know your cache is healthy?10
Memory card and where the analogy breaks11
Practice: five levels12
Interview questions13
Solutions14
Sources15
Code lab13. Caching at Scale
13.8 How do you know your cache is healthy?
3 min read+10 XP
At the wedding: The head cook watches a few things: how often the fridge has what's asked for, how often things are thrown out for space, how full it is, and how long the queue at the shop is getting.
In the system: Watch these, per cluster and per key prefix (Chapter 30):
| Metric | What it tells you | Worry when |
|---|---|---|
| Hit rate (overall and per prefix) | Is the cache doing its job? | It drops suddenly (bad deploy changed keys, restart) |
| Evictions per second | Is it big enough? | High evictions together with a falling hit rate |
| Memory used | Room to grow? | Close to the limit |
| Latency (p50, p99) | Is the cache itself slow? | p99 rising: hot key, big values, network |
| Per-node load | Is load even? | One node far busier than others: hot key (13.4) |
| Database queries per second | What the cache is saving | Jumps while the hit rate falls: stampede or avalanche |
| Connections, errors, timeouts | Can clients reach it? | Rising errors: failover, overload |
Redis, for example, reports hits and misses (keyspace_hits, keyspace_misses), evicted keys and memory use in its INFO output.
Alerts worth having: hit rate drops by a large amount in a few minutes; one node's load is several times the average; evictions spike; database queries jump while the cache's hit rate falls.
Memory code: "Watch the hits, the throw-outs, the fullness and the shop's queue."