32. Design a Rate Limiter as a Service
1
The interview prompt2
32.1 What do you ask first?3
32.2 How big is this, really?4
32.3 What are the doors?5
32.4 What do we store?6
32.5 What does the whole system look like?7
32.6 How do you pick the algorithm?8
32.7 How do counters work in Redis?9
32.8 What race conditions do you have to handle?10
32.9 How do limits work across regions?11
32.10 What would you say in the last five minutes?12
Score yourself13
Variations to practise14
Sources15
Code lab32. Design a Rate Limiter as a Service
Score yourself
3 min read+10 XP
Score yourself
Give yourself 0, 1 or 2 points per row.
| Area | 0 points | 1 point | 2 points |
|---|---|---|---|
| Requirements | Started drawing | Listed features | Features, latency budget, fail-open/closed and multi-region agreed and written down |
| Estimates | None | Numbers, not used later | Numbers that drove decisions (ops/s → shards, memory is small, latency budget) |
| API and data | Skipped | Check API or rule model, not both | Both, with remaining/retry-after and Redis key design |
| High-level design | Missing pieces | Boxes, no walkthrough | Check path and rule-change path walked, failure handling included |
| Deep dive | Hand-waved | One algorithm described | Algorithm per rule, atomic script, multi-rule race or regions handled |
| Trade-offs | None named | Named when asked | Raised unprompted (library vs sidecar, exact vs fast, open vs closed) |
| Communication | Long silences | Mostly clear | Thought out loud, checked in, managed time |
12 to 14: strong hire signal. 8 to 11: solid, polish the weak rows. Under 8: redo the round in a week, timer on.
The bar: fresher vs SDE-2
| Fresher | SDE-2 | |
|---|---|---|
| Algorithm | Explains token bucket or fixed window clearly | Picks per rule, knows boundary bursts, costs and retry-after |
| Shared state | Puts a counter in Redis | One atomic script, TTLs, sharding, hash tags, Redis clock |
| Concurrency | Knows INCR is atomic | Handles multi-rule checks without leaking tokens |
| Failure | "Redis has replicas" | Timeouts, breaker, fail-open vs closed per rule, local backstop |
| Regions | Not mentioned | Local enforcement with rebalanced shares; home region for tight limits |
Common mistakes
- A network hop to a separate limiter service on every request without discussing latency.
- GET, check, SET instead of one atomic step.
- No plan for the limiter being down, so a Redis blip becomes a site-wide outage.
- One global counter across regions on the hot path.
- Checking several rules one by one and silently spending tokens on denied requests.
- No safeguards on rule changes: one typo throttles every customer.
Quick check
Five questions on this chapter. Pick an answer, then check it.
quiz