32. Design a Rate Limiter as a Service
1
The interview prompt2
32.1 What do you ask first?3
32.2 How big is this, really?4
32.3 What are the doors?5
32.4 What do we store?6
32.5 What does the whole system look like?7
32.6 How do you pick the algorithm?8
32.7 How do counters work in Redis?9
32.8 What race conditions do you have to handle?10
32.9 How do limits work across regions?11
32.10 What would you say in the last five minutes?12
Score yourself13
Variations to practise14
Sources15
Code lab32. Design a Rate Limiter as a Service
32.4 What do we store?
3 min read+10 XP
rules (rules service, a small SQL table; cached in every limiter client)
rule_id "partner-api"
key_type api_key | user | ip | account | custom
algorithm token_bucket | sliding_window | fixed_window (daily quotas)
limit, window 100 per 60 s
burst 100
cost_overrides { "/search": 5 }
on_failure open | closed
owner_team, version, updated_at
buckets (Redis, one hash per rule + key, with a TTL)
key: rl:{partner-api:apikey_42}
value: tokens = 37.4, ts = 1728381600.512
TTL: 2 × window, so idle keys disappear on their own
Interviewer pushback:
- "Why braces in the Redis key?" → They're Redis Cluster hash tags: only the part inside braces is hashed, so we control which slot a key lands on. That matters when one script must touch several keys (32.8).
- "Why cache the rules in the client?" → Rules change rarely and are read on every check. Fetching them per request would double the round trips. Clients poll or subscribe for changes and apply new versions within seconds.
Memory code: "Rules are read everywhere and changed rarely; counters are tiny and hot."