Art of Codeartofcode
Lesson 11 of 15

32. Design a Rate Limiter as a Service

1
The interview prompt
2
32.1 What do you ask first?
3
32.2 How big is this, really?
4
32.3 What are the doors?
5
32.4 What do we store?
6
32.5 What does the whole system look like?
7
32.6 How do you pick the algorithm?
8
32.7 How do counters work in Redis?
9
32.8 What race conditions do you have to handle?
10
32.9 How do limits work across regions?
11
32.10 What would you say in the last five minutes?
12
Score yourself
13
Variations to practise
14
Sources
15
Code lab
32. Design a Rate Limiter as a Service

32.10 What would you say in the last five minutes?

3 min read+10 XP

Step 6: Wrap-up (5 minutes)

  • Bottleneck: Redis operations. Covered by sharding, one script per check, and local token batches for giant tenants.
  • Single points of failure: Redis shards (replicas, plus fail-open with local backstops); the rules service (clients keep the last good rules in memory, so it can be down without affecting checks).
  • Safety: rule changes are versioned, validated (no "limit 0 for everyone" by typo), rolled out gradually, and instantly reversible. A bad rule is the most likely way this service hurts the company.
  • Visibility: per-rule dashboards of allowed vs limited, top limited keys, and alerts when a rule starts limiting far more than usual.
  • Next with more time: cost-based limits for GraphQL, concurrency limits (in-flight requests rather than rate), self-serve rule testing ("what would this rule have blocked yesterday?").

Memory code: "The limiter must never be the outage."