Art of Codeartofcode
Lesson 10 of 15

32. Design a Rate Limiter as a Service

1
The interview prompt
2
32.1 What do you ask first?
3
32.2 How big is this, really?
4
32.3 What are the doors?
5
32.4 What do we store?
6
32.5 What does the whole system look like?
7
32.6 How do you pick the algorithm?
8
32.7 How do counters work in Redis?
9
32.8 What race conditions do you have to handle?
10
32.9 How do limits work across regions?
11
32.10 What would you say in the last five minutes?
12
Score yourself
13
Variations to practise
14
Sources
15
Code lab
32. Design a Rate Limiter as a Service

32.9 How do limits work across regions?

3 min read+10 XP

The company runs in Mumbai, Frankfurt and Virginia. A partner's limit is 900 a minute globally. Options:

  1. One global counter in one region: exact, but every check in the other regions crosses an ocean (100+ ms) and fails when that region does. Rejected for the hot path.
  2. Split the limit evenly, 300 per region, each enforced locally: fast and independent, but wasteful when traffic is uneven.
  3. Local enforcement with shares rebalanced by demand: each region enforces its share locally; a small job re-divides shares every few seconds from recent demand.
  4. Home-region routing: send all checks for a key to its home region. Exact, used for small, security-critical limits.

splitting 900 evenly serves 600 of 900 wanted; rebalancing by demand serves 870.

Figure: even splits waste the limit where traffic is light and refuse it where traffic is heavy.

In the chapter's code, with demand of 600 in Mumbai, 200 in Frankfurt and 100 in Virginia, an even split serves only 600 of the 900 wanted. Rebalancing shares to 570 / 210 / 120 (keeping a small floor in each region) serves 870, while the total never exceeds 900.

Our pick: option 3 for API limits (fast, survives a region outage, slightly over or under for a few seconds after shifts), and option 4 for login and OTP rules, which are small and must be tight.

Interviewer pushback:

  • "What if the rebalancing job is down?" → Shares freeze at their last values; enforcement continues. If a region loses contact entirely, its share stays usable, so the global limit may be exceeded briefly; that's the price of not making every check depend on another continent.

Memory code: "Enforce locally, rebalance often, route the tight ones home."