32. Design a Rate Limiter as a Service
1
The interview prompt2
32.1 What do you ask first?3
32.2 How big is this, really?4
32.3 What are the doors?5
32.4 What do we store?6
32.5 What does the whole system look like?7
32.6 How do you pick the algorithm?8
32.7 How do counters work in Redis?9
32.8 What race conditions do you have to handle?10
32.9 How do limits work across regions?11
32.10 What would you say in the last five minutes?12
Score yourself13
Variations to practise14
Sources15
Code lab32. Design a Rate Limiter as a Service
32.6 How do you pick the algorithm?
3 min read+10 XP
Step 5: Deep dives (10 minutes)
The service supports a few algorithms and lets each rule choose (Chapter 25):
| Rule | Algorithm | Why |
|---|---|---|
| Partner API: 100 a minute, bursts OK | Token bucket (capacity 100, refill 100/min) | Bursty clients, two numbers per key, supports per-call cost |
| Search: expensive | Token bucket with cost 5 | Same bucket, weighted calls |
| Daily quota: 10,000 a day | Fixed window per calendar day | Simple; a boundary burst over a day barely matters |
| High-volume per-key limit needing accuracy | Sliding window counter | Two counters, close to exact |
| Login: 5 per 15 minutes per account | Sliding log (or token bucket with capacity 5) | Tiny, must be tight |
| Calls to a fragile partner | Leaky bucket in the outbound client | Smooth output, not just a cap |
Default: token bucket. It handles bursts, needs two numbers per key, gives a natural retry_after ((cost − tokens) ÷ refill rate), and supports costs. In the chapter's code, a bucket of 100 allows exactly 100 of 101 instant calls, reports retry after 1 s, and allows the next call 0.6 s later.
Interviewer pushback:
- "Why not just fixed windows everywhere? They're simplest." → The boundary burst: 2× the limit in a second at the window edge (Chapter 25). Fine for daily quotas, bad for protecting a service per minute.
Memory code: "Bucket by default; pick another only for a named reason."