32. Design a Rate Limiter as a Service
1
The interview prompt2
32.1 What do you ask first?3
32.2 How big is this, really?4
32.3 What are the doors?5
32.4 What do we store?6
32.5 What does the whole system look like?7
32.6 How do you pick the algorithm?8
32.7 How do counters work in Redis?9
32.8 What race conditions do you have to handle?10
32.9 How do limits work across regions?11
32.10 What would you say in the last five minutes?12
Score yourself13
Variations to practise14
Sources15
Code lab32. Design a Rate Limiter as a Service
32.2 How big is this, really?
3 min read+10 XP
Step 2: Estimates (5 minutes)
Assume about 100 bytes per bucket in Redis (key, tokens, timestamp, overhead) and one Redis operation per check.
| What | Working | Result |
|---|---|---|
| Checks a second at peak | given | 200,000 |
| Redis operations a second | 200,000 × 1 script call | 200,000 |
| Redis shards | 200,000 ÷ 50,000 (assumed safe ops per shard, leaving headroom) | 4 shards, each with a replica |
| Memory for buckets | 10 million × 100 bytes | ≈ 1 GB: tiny |
| Network | 200,000 × ~200 bytes request + reply | ≈ 40 MB/s: fine |
| Added latency | one same-zone round trip + script | ≈ 0.5-1 ms typical |
What the numbers say, out loud:
- "Memory is trivial. 1 GB of counters. Storage isn't the problem."
- "200,000 operations a second is the real load, so the counter store must be sharded and every check must be one round trip."
- "Latency matters more than size: the limiter sits on every request, so it must be in the same zone as the callers, and callers need tight timeouts."
Interviewer pushback:
- "Why 50,000 operations per shard when Redis can do much more?" → Lua scripts cost more than a plain GET, traffic spikes, and we want room to lose a shard. It's an assumption to check with a load test.
- "Could we skip Redis and count in each server's memory?" → Only as a backstop. With a load balancer spreading one client across many servers, local counts can't enforce a shared limit (Chapter 25's 55-of-100 example).
Memory code: "Tiny memory, huge rate, one round trip."