Art of Codeartofcode
Lesson 15 of 16

12. Caching: The Basics

1
The picture to keep in your head
2
12.1 What is a cache, and what are a hit, a miss and the hit rate?
3
12.2 Why is a cache faster than the database?
4
12.3 Where does a cache physically live?
5
12.4 On a miss, who fetches the data?
6
12.5 When the data changes, what gets written where?
7
12.6 What happens when the real data changes, and why add jitter?
8
12.7 What happens when the cache is full?
9
12.8 How do you invalidate safely, and what's the race everyone hits?
10
12.9 When should you cache, what should you never cache, and how do you name keys?
11
Memory card and where the analogy breaks
12
Practice: five levels
13
Interview questions
14
Solutions
15
Sources
16
Code lab
12. Caching: The Basics

Sources

3 min read+10 XP

Source notes

  • RAM ~100 ns and data-centre round trip ~500 µs: Jeff Dean, "Latency numbers every programmer should know" (approximate; vary by hardware and year).
  • Level 2 arithmetic and Level 3 solution code: computed and tested (code/ch12_caching.py, all checks pass, 2026-10-08). The core of this chapter (sections on what a cache is, why it's faster, where it lives, staleness, eviction, when to cache, and cost, plus Levels 1 to 5 and Q1 to Q12) is the approved v4 text from 2026-10-01, reorganised into the locked table of contents.
  • Redis reports cache hits and misses in its INFO stats (keyspace_hits, keyspace_misses) and supports configurable eviction policies, including LRU-like and LFU-like policies, and optional persistence (Redis documentation).
  • Caffeine is a high-performance in-process cache library for Java (project documentation).
  • Jitter figures (10,000 keys; busiest second 208 with ±30 s) and the LRU vs LFU scan example come from this chapter's code, run 2026-10-08.