Courses

System DesignHigh-level system design, one everyday scene at a time: 30 concepts and 15 interview rounds.712 lessons

All courses Tutorials My Learning

Categories

Read & write

All storiesEvery post, newest first.Write for usPublish your stories and earn from what you write.Writer leaderboardsWho is writing the most read posts.

Categories

Featured

JSON FormatterFormat, validate and minify JSON.Regex TesterTest patterns against live text.Merge PDFCombine PDFs in your browser.JWT DecoderRead a token’s header and claims.Password GeneratorStrong passwords, generated locally.Gradient GeneratorBuild CSS gradients visually.

Play a round

DevdleOne developer word a day, six guesses.Python RushClassic snake, with coding icons to eat.Blink or BreakA quick test of your reflexes.

Learn with us

Live CoursesInstructor-led courses. Join a waitlist and be first in when one starts.LearningStructured programming courses with AI assistance and certificates.

System Design

12. Caching: The Basics

Lesson 13 of 16

12. Caching: The Basics

The picture to keep in your head

12.1 What is a cache, and what are a hit, a miss and the hit rate?

12.2 Why is a cache faster than the database?

12.3 Where does a cache physically live?

12.4 On a miss, who fetches the data?

12.5 When the data changes, what gets written where?

12.6 What happens when the real data changes, and why add jitter?

12.7 What happens when the cache is full?

12.8 How do you invalidate safely, and what's the race everyone hits?

12.9 When should you cache, what should you never cache, and how do you name keys?

Memory card and where the analogy breaks

Practice: five levels

Interview questions

12. Caching: The Basics

Interview questions

11 min read+10 XP

Interview questions (with the counter-questions you'll get)

Interviewers rarely stop at your first answer. For each question below: what they're really testing, a strong answer, the counter-questions they're likely to throw back, and the trap to avoid.

Q1. What is caching, and why would you use it?

They're testing: whether you understand it's about copies, speed and load, not just "makes it fast".

Think about your answer first, then open it.

reveal

Q2. What's a cache hit, a miss, and a hit rate? What's a good hit rate?

They're testing: precise definitions and whether you can reason with numbers.

Think about your answer first, then open it.

reveal

Q3. Why is a cache faster than a database?

They're testing: whether you know why, not just that.

Think about your answer first, then open it.

reveal

Q4. Where would you put a cache in a typical web app?

They're testing: whether you see the whole request path, not just "add Redis".

Think about your answer first, then open it.

reveal

Q5. Walk me through the cache-aside pattern.

They're testing: that you can describe the flow precisely, including the miss path.

Think about your answer first, then open it.

reveal

Q6. How do you keep the cache consistent with the database?

They're testing: whether you know the real-world bug.

Think about your answer first, then open it.

reveal

Q7. How do you choose a TTL?

They're testing: trade-off thinking.

Think about your answer first, then open it.

reveal

Q8. What happens when the cache runs out of memory?

They're testing: eviction policies.

Think about your answer first, then open it.

reveal

Q9. What should you never cache?

They're testing: judgement and safety.

Think about your answer first, then open it.

reveal

Q10. Is the cache a source of truth? What happens if Redis restarts?

They're testing: whether you design for the cache disappearing.

Think about your answer first, then open it.

reveal

Q11. You added a cache, but the database load barely dropped. Why?

They're testing: debugging instinct.

Think about your answer first, then open it.

reveal

Q12. How do you know your cache is working well?

They're testing: operational maturity.

Think about your answer first, then open it.

reveal

Q13. Write-through, write-back or write-around: which would you use?

They're testing: write-path trade-offs.

Think about your answer first, then open it.

reveal

Q14. Cache-aside or read-through?

They're testing: knowing that both exist and why you'd choose.

Think about your answer first, then open it.

reveal

No tracks available

Lofi Player

1Show the strong answer, counter-questions and trap
2**Strong answer:** A cache keeps copies of frequently read data in fast storage, usually memory, close to the app. It cuts response time because you skip the slow trip to the database, and it cuts database load because repeated questions are answered once. The trade-off is that the copy can be stale and you add another system.
3
4**Counter-questions:**
5- *"So why not cache everything?"* → Memory is limited and costly, most caches lose data on restart, and a cache can only look things up by key, not run queries. Cache the hot, read-heavy, slightly-stale-is-fine data.
6- *"When would a cache make things worse?"* → When the hit rate is low (every request is unique), when data changes constantly, or when the extra complexity causes stale-data bugs in places that must be exact.
7
8**Trap:** saying only "it makes things faster". Always mention database load and staleness.

1Show the strong answer, counter-questions and trap
2**Strong answer:** A hit is when the exact key you asked for is in the cache and hasn't expired. A miss is when it isn't, or it has expired, so you go to the source. Hit rate is hits ÷ total lookups. "Good" depends on the workload, but for read-heavy data, teams usually aim for 90% or higher, because the misses dominate the average latency.
3
4**Counter-questions:**
5- *"Why does going from 90% to 99% matter so much?"* → With a 1 ms cache and 50 ms database, 90% gives about 5.9 ms on average and 99% gives about 1.5 ms. Cutting misses from 10% to 1% makes requests about 4 times faster and cuts database traffic by 10 times.
6- *"Is an expired key a hit or a miss?"* → A miss. The data may still be sitting in memory, but it's not valid, so you go to the source.
7- *"How would you measure hit rate in production?"* → Count hits and misses in the app or read the cache's own stats (Redis reports keyspace hits and misses), and track it over time on a dashboard.
8
9**Trap:** defining a hit as "the cache has data".

1Show the strong answer, counter-questions and trap
2**Strong answer:** Two reasons. It keeps data in RAM, which is orders of magnitude faster than going over the network and to disk. And it does a much simpler job: an exact-key lookup, with no query parsing, planning, joins or disk reads.
3
4**Counter-questions:**
5- *"But Redis runs on another server. Isn't that a network trip too?"* → Yes, one hop. It's still much faster because the lookup itself is tiny and everything is already in memory. An in-process cache avoids even that hop.
6- *"When would a cache not be faster?"* → If the database already serves that data from its own memory with a simple indexed lookup and the cache adds a network hop, the gain can be small. Caches shine when the underlying work is expensive.
7
8**Trap:** saying "because it's Redis". Explain RAM and the simpler operation.

1Show the strong answer, counter-questions and trap
2**Strong answer:** At several layers. Browser cache and CDN for static files and public content, a shared cache like Redis for data all app servers need, sometimes an in-process cache for tiny hot data like config, and the database's own buffer cache underneath. Cache as close to the user as the data allows.
3
4**Counter-questions:**
5- *"In-process or Redis: which would you pick?"* → In-process for small, rarely changing data where each server having its own copy is fine. Redis when servers must agree on one copy or the data is too big to duplicate on every server.
6- *"Can you put user-specific data in a CDN?"* → Generally no. CDNs serve the same response to many users. Personal data belongs in a per-user cache like Redis, keyed by user, or not cached at all.
7
8**Trap:** naming only one layer.

1Show the strong answer, counter-questions and trap
2**Strong answer:** The app checks the cache first. On a hit it returns the value. On a miss it reads from the database, writes a copy into the cache with a TTL, and returns it. Only data that's actually requested gets cached.
3
4**Counter-questions:**
5- *"What happens if the cache server is down?"* → With cache-aside the app can fall back to the database, so things keep working, just slower. But the database suddenly gets all the traffic, so you need timeouts on the cache call and protection on the database (rate limits, connection limits).
6- *"What if 1,000 requests miss the same key at the same moment?"* → They'd all hit the database together (a stampede). Use a lock or lease so one request refills the cache while the rest wait, or serve the slightly old value while one request refreshes it (Chapter 13).
7
8**Trap:** forgetting to mention setting a TTL when filling the cache.

1Show the strong answer, counter-questions and trap
2**Strong answer:** On every write, update the database first, then delete the cache key, so the next read loads the fresh value. Keep a TTL as a safety net in case a delete is missed. For data that must be exact, like payment amounts, read from the database instead of the cache.
3
4**Counter-questions:**
5- *"Why delete the key instead of updating it with the new value?"* → Deleting is simpler and safer. If two writes race, updating the cache can leave the older value in place; deleting just forces a fresh read.
6- *"Why database first, then cache? What if you do it the other way round?"* → If you delete the cache first, another request can miss in the gap, read the old value from the database, and put it back in the cache, where it stays until the TTL ends. That's exactly the Level 4 bug.
7- *"What if the delete fails?"* → The TTL eventually fixes it. For important data, retry the delete, or publish change events that a separate worker uses to clear the cache.
8
9**Trap:** saying "the cache is always in sync". Say how long it can be stale and what limits that window.

1Show the strong answer, counter-questions and trap
2**Strong answer:** Ask how stale the data can be before it hurts a user. Short TTLs give fresher data but more misses and more database load; long TTLs give more hits but staler data. Rarely changing data gets hours, prices during a sale get seconds plus invalidation, and must-be-exact data isn't cached.
3
4**Counter-questions:**
5- *"What if you set the same TTL on thousands of keys at once?"* → They all expire together and the database gets flooded. Add random jitter to TTLs to spread expiries out.
6- *"Can you rely only on TTL?"* → For data where a few minutes of staleness is fine, yes. For data users notice changing (prices, profiles), combine TTL with delete-on-change.
7
8**Trap:** giving one TTL number without explaining why.

1Show the strong answer, counter-questions and trap
2**Strong answer:** It evicts entries to make room, based on a policy. LRU (least recently used) is the common default: remove what hasn't been accessed in the longest time. LFU (least frequently used) keeps items that are popular over a long period. You can also evict based on TTL, or refuse new writes. Redis lets you configure this.
3
4**Counter-questions:**
5- *"When does LRU make a bad choice?"* → When a one-off scan reads lots of items once. They become "recently used" and push out genuinely popular items. LFU handles that better.
6- *"What metric tells you the cache is too small?"* → A high eviction rate together with a falling hit rate.
7
8**Trap:** not knowing that caches evict at all.

1Show the strong answer, counter-questions and trap
2**Strong answer:** Anything that must be exactly right at that moment: an account balance during payment, the final stock check before confirming an order, permissions right after they're revoked. Also data that's unique per request (no reuse), and data that changes faster than it's read.
3
4**Counter-questions:**
5- *"Can you cache user-specific data?"* → Yes, if the key includes the user's ID and access is controlled. The big danger is a key mistake that serves one user's data to another. That's a privacy incident, not just a bug.
6- *"So how does a product page show stock without hitting the database every time?"* → Show an approximate, briefly cached stock level ("only a few left"), and do the exact check against the database at checkout.
7
8**Trap:** "cache everything that's slow."

1Show the strong answer, counter-questions and trap
2**Strong answer:** No. The database is the source of truth and the cache is a disposable copy. If Redis restarts empty, every request misses for a while, the database takes the extra load, and the cache refills. That's why you never store something only in the cache.
3
4**Counter-questions:**
5- *"Redis can persist to disk. Doesn't that make it safe as the source of truth?"* → Redis does offer persistence options, but most cache setups treat it as disposable for speed and simplicity. Using it as a primary store is a separate design decision with its own durability trade-offs.
6- *"How do you stop the database being flattened after a restart?"* → Warm the cache with the hottest keys before taking traffic, limit concurrent database queries, and let the cache fill gradually (Chapter 13).
7
8**Trap:** assuming the cache will always be there.

1Show the strong answer, counter-questions and trap
2**Strong answer:** Likely causes: a low hit rate because keys are too specific (for example, including a timestamp or a full query string in the key), TTLs too short, the cache too small so items are evicted before reuse, the traffic being mostly writes or unique requests, or a stampede on popular keys. Check hit rate, eviction count and the top missed keys first.
3
4**Counter-questions:**
5- *"How would you prove which one it is?"* → Look at hit rate per key pattern, evictions per second, and the TTL distribution, and sample the most frequent misses.
6- *"Hit rate is 95%, but pages are still slow. Now what?"* → The slowness is somewhere else: large values being serialised and deserialised, slow misses on a few heavy keys, the network to the cache, or work after the cache lookup. Measure each step.
7
8**Trap:** guessing without saying how you'd measure.

1Show the strong answer, counter-questions and trap
2**Strong answer:** Track hit rate, latency (average and slow tail, like p99) for cache and database, eviction rate, memory use, and database queries per second before and after. A healthy cache shows a high, stable hit rate and visibly lower database load.
3
4**Counter-questions:**
5- *"What alert would you set?"* → A sudden drop in hit rate (could mean a bad deploy changed the keys, or the cache restarted), a spike in evictions, or the cache's memory close to full.
6
7**Trap:** only mentioning hit rate.

1Show the strong answer, counter-questions and trap
2**Strong answer:** Write-around with cache-aside is the safe default: write the database, delete the cached copy, and let the next read refill it. Write-through suits data that's read right after it's written, at the cost of slower writes and caching things nobody reads. Write-back gives the fastest writes and batches them, but a crash before the flush loses data, so I'd only use it for things like counters and metrics, never orders or payments.
3
4**Counter-questions:**
5- *"How would you make write-back safer?"* → Flush often, keep the cache replicated and persistent, and design so losing a few seconds of updates is acceptable. If it isn't acceptable, don't use write-back.
6
7**Trap:** recommending write-back for anything important because it's fast.

1Show the strong answer, counter-questions and trap
2**Strong answer:** In cache-aside, the application checks the cache, loads from the database on a miss, and fills the cache. In read-through, the app only talks to the cache, which loads missing data itself. Read-through keeps loading logic in one place; cache-aside gives the app full control and keeps working if the cache is down. Many teams wrap cache-aside in a small helper to get the best of both.
3
4**Counter-questions:**
5- *"Which handles a cache outage better?"* → Cache-aside, since the app already knows how to read the database directly.
6
7**Trap:** describing only one pattern.
8
9---
10
11## Practice coverage
12
13Every section in this chapter is tested at least once.
14
15| Section | Tested by |
16|---|---|
17| 12.1 What a cache is, hits and misses | Level 2, Level 3, Q1, Q2, Q10 |
18| 12.2 Why it's faster | Q3 |
19| 12.3 Where caches live | Level 5.2, Q4 |
20| 12.4 Read strategies | Level 3, Q5, Q14 |
21| 12.5 Write strategies | Level 1.6, Level 3 extension, Q13 |
22| 12.6 TTL and jitter | Level 1, Level 2.4, Level 3, Level 5.3, Q7 |
23| 12.7 Eviction | Level 3, Q8 |
24| 12.8 Invalidation and the race | Level 4, Level 5.4, Q6 |
25| 12.9 When to cache, cost, keys | Level 1, Level 5.1, Q9, Q11, Q12 |
26
27---