12. Caching: The Basics
1
The picture to keep in your head2
12.1 What is a cache, and what are a hit, a miss and the hit rate?3
12.2 Why is a cache faster than the database?4
12.3 Where does a cache physically live?5
12.4 On a miss, who fetches the data?6
12.5 When the data changes, what gets written where?7
12.6 What happens when the real data changes, and why add jitter?8
12.7 What happens when the cache is full?9
12.8 How do you invalidate safely, and what's the race everyone hits?10
12.9 When should you cache, what should you never cache, and how do you name keys?11
Memory card and where the analogy breaks12
Practice: five levels13
Interview questions14
Solutions15
Sources16
Code lab12. Caching: The Basics
12.3 Where does a cache physically live?
3 min read+10 XP
In the kitchen: Two options. A snack in your pocket (instant, but only you can use it, and it's gone when you change clothes) or a shared pantry for the whole flat (everyone uses the same shelf, but you take a few steps to reach it).
In the system:
| Kitchen | Where | Example | Good | Not so good |
|---|---|---|---|---|
| Inside your app's memory | A dictionary, or a library like Caffeine | Fastest, no network | Each app server has its own separate copy; lost when the app restarts | |
| Pantry | On a separate cache server | Redis, Memcached | One shared copy for all your app servers | One network hop; one more system to run |
Those are the two you build yourself. Along a real request there are more layers, from closest to the user to closest to the data:
Figure: caches at every layer. Cache as close to the user as the data allows.
- Browser: keeps copies of images, scripts and some API responses on the user's device (HTTP caching, Chapter 13).
- CDN edge: keeps public content near users worldwide (Chapter 26).
- Pocket and pantry: your app's own caches.
- The database's buffer cache: the database keeps hot pages in its own memory. You get it for free, but it still pays for parsing and planning every query (12.2).
Memory code: "Pocket or Pantry." In-app cache = your pocket (fast, private). Redis = the shared pantry (shared, a short walk).