13. Caching at Scale
13.2 What is cache penetration?
At the wedding: Guests keep asking for "mutton biryani", which isn't on the menu. Every time, a helper checks the fridge (not there), then runs to the shop to check (not there either), and comes back empty-handed. Nothing is ever stored, so every request is a wasted trip.
In the system: Cache penetration is when requests ask for data that doesn't exist. The cache misses (nothing to cache), the database says "not found", and because nothing is stored, every such request goes to the database. It happens with typos and broken links, but also with bots and attackers requesting random IDs (/product/zz91, /product/zz92...) on purpose.
Figure: without protection, every lookup for a missing product reaches the database.
Three shields:
- Negative caching: cache the "not found" answer too, with a short TTL (seconds to a minute), so a real product created a moment later isn't hidden for long. In the chapter's code, 1,000 lookups of a missing product cause 1,000 database reads without it and 1 with it.
- A Bloom filter in front: a compact structure that can say "this ID definitely doesn't exist" instantly, using very little memory (Chapter 20). Requests for impossible IDs never reach the cache or the database.
- Validate and rate-limit: reject IDs in the wrong format before any lookup, and slow down clients making many not-found requests (Chapter 25).
Memory code: "Write 'we don't sell that' on the fridge door."