Art of Codeartofcode
Lesson 13 of 15

32. Design a Rate Limiter as a Service

1
The interview prompt
2
32.1 What do you ask first?
3
32.2 How big is this, really?
4
32.3 What are the doors?
5
32.4 What do we store?
6
32.5 What does the whole system look like?
7
32.6 How do you pick the algorithm?
8
32.7 How do counters work in Redis?
9
32.8 What race conditions do you have to handle?
10
32.9 How do limits work across regions?
11
32.10 What would you say in the last five minutes?
12
Score yourself
13
Variations to practise
14
Sources
15
Code lab
32. Design a Rate Limiter as a Service

Variations to practise

3 min read+10 XP

Chapters used

Approach (1), Estimation (2), Caching at scale and Redis Cluster (13), Transactions and races (17), APIs (24), Rate limiting algorithms (25), Microservices and sidecars (27), Failures and circuit breakers (29), Observability (30).

Try these variations

Each is a common follow-up. Answer in two minutes, then check the sketch.

  1. "Limit concurrent requests, not rate." → Track in-flight requests per key: increment on start, decrement on finish (with a TTL in case a caller dies mid-request). Good for expensive, long-running endpoints like report exports.
  2. "Bill customers per request beyond their quota." → Billing needs exact counts: write usage events to a queue and aggregate them durably (Chapter 22). Use the limiter for enforcement and the event pipeline for the bill.
  3. "10× the traffic: 2 million checks a second." → 40 shards at the same per-shard load, local token batches for big tenants, and checks in the gateway or sidecar only; reconsider whether every endpoint needs a central check or whether coarse local limits suffice for some.
  4. "Give each customer a dashboard of their usage." → Emit allowed/limited counts per key to an analytics store (asynchronously, aggregated per minute), not by querying Redis counters.