Art of Codeartofcode
Lesson 14 of 15

13. Caching at Scale

1
The picture to keep in your head
2
13.1 What is a cache stampede, and how do you stop one?
3
13.2 What is cache penetration?
4
13.3 What is a cache avalanche?
5
13.4 What is a hot key, and why can't sharding fix it?
6
13.5 How do you spread a cache across many machines?
7
13.6 What is cache warming?
8
13.7 How does HTTP caching work in browsers and CDNs?
9
13.8 How do you know your cache is healthy?
10
Memory card and where the analogy breaks
11
Practice: five levels
12
Interview questions
13
Solutions
14
Sources
15
Code lab
13. Caching at Scale

Sources

3 min read+10 XP

Source notes

  • Leases to prevent thundering herds and stale sets: Rajesh Nishtala et al., "Scaling Memcache at Facebook", NSDI 2013.
  • Probabilistic early expiration: Andrea Vattani, Flavio Chierichetti and Keegan Lowenstein, "Optimal Probabilistic Cache Stampede Prevention", Proceedings of the VLDB Endowment, 2015.
  • Redis Cluster: 16,384 hash slots, CRC16 key hashing (reference value 0x31C3 for "123456789"), hash tags, replicas and failover: Redis Cluster specification and documentation. Redis INFO fields and hot-key reporting with LFU eviction: Redis documentation.
  • HTTP caching (Cache-Control, ETag, 304, Vary): RFC 9111 and RFC 9110. stale-while-revalidate: RFC 5861.
  • All traffic and sizing numbers are illustrative assumptions; arithmetic computed 2026-10-08.
  • Level 3 and Level 4 code: tested 2026-10-08 (code/ch13_cache_at_scale.py, all checks pass).