Art of Codeartofcode
Lesson 4 of 15

13. Caching at Scale

1
The picture to keep in your head
2
13.1 What is a cache stampede, and how do you stop one?
3
13.2 What is cache penetration?
4
13.3 What is a cache avalanche?
5
13.4 What is a hot key, and why can't sharding fix it?
6
13.5 How do you spread a cache across many machines?
7
13.6 What is cache warming?
8
13.7 How does HTTP caching work in browsers and CDNs?
9
13.8 How do you know your cache is healthy?
10
Memory card and where the analogy breaks
11
Practice: five levels
12
Interview questions
13
Solutions
14
Sources
15
Code lab
13. Caching at Scale

13.3 What is a cache avalanche?

3 min read+10 XP

At the wedding: Two different disasters. Every carton was bought on the same morning with the same use-by date, so the whole fridge goes off at once. Or the fridge's power cuts out, and everything in it is gone. Either way, the entire wedding runs to the shop together.

In the system: A cache avalanche is when many keys miss at the same time, so a huge share of traffic falls on the database at once. Two causes:

  1. Mass expiry: lots of keys were cached at the same moment with the same TTL (after a deploy, a restart, or a nightly warm-up), so they all expire together. Fix: jitter (Chapter 12, section 12.6): add a random spread to every TTL. If 1 million keys expire within the same 10 minutes, that's about 1,700 extra misses a second for 10 minutes; spread across an hour, it's barely noticeable.
  2. The cache itself goes down: a Redis node or the whole cluster restarts empty. Now every request is a miss. Fixes:
    • Replicas for every cache node, so one failure doesn't empty anything (13.5).
    • Persistence or a warm restart, so a restarted node comes back with its data.
    • Protect the database: strict connection and query limits, timeouts on cache calls, a circuit breaker (Chapter 29) and rate limiting (Chapter 25), so it degrades instead of collapsing.
    • Serve something: stale data from a local "pocket" cache, or a simplified page, rather than nothing.
    • Warm up before taking full traffic (13.6).

The difference from a stampede: a stampede is one key with many requests; an avalanche is many keys at once.

Memory code: "Don't let the whole fridge go off, or switch off, at once."