Art of Codeartofcode
Lesson 9 of 15

32. Design a Rate Limiter as a Service

1
The interview prompt
2
32.1 What do you ask first?
3
32.2 How big is this, really?
4
32.3 What are the doors?
5
32.4 What do we store?
6
32.5 What does the whole system look like?
7
32.6 How do you pick the algorithm?
8
32.7 How do counters work in Redis?
9
32.8 What race conditions do you have to handle?
10
32.9 How do limits work across regions?
11
32.10 What would you say in the last five minutes?
12
Score yourself
13
Variations to practise
14
Sources
15
Code lab
32. Design a Rate Limiter as a Service

32.8 What race conditions do you have to handle?

3 min read+10 XP

GET then SET lets 20 through instead of 5; checking two rules one by one spends tokens from the first even when the second denies.

Figure: read-decide-write must be one step, and so must checking several rules for one request.

Race 1: read, decide, write. If the client does GET, decides, then SET, concurrent checks all read the same old value. In the chapter's code, 20 checks arriving with 95 of 100 used: read-then-write allows all 20; the atomic script allows exactly 5. (Chapter 17's lost update, Chapter 25's Level 4.)

Race 2: several rules for one request. A login is checked against per-IP (50) and per-account (5). Checking them one at a time spends a token from the first rule even when the second denies. In the chapter's code, with the account already used up, 10 denied attempts still drain the IP bucket from 50 to 40; one script that checks both and spends only if both allow leaves it at 50. Over time, the naive version locks out innocent users who share an IP.

One catch: in Redis Cluster, a multi-key script needs all its keys in the same slot, and an IP key and an account key normally hash to different slots (forcing them together with one hash tag would break per-account counting across IPs). The practical options: keep the small, security-critical rules (login, OTP) in their own non-sharded Redis with a replica, where one script can touch both keys; or check rules in sequence and refund the earlier tokens when a later rule denies.

Race 3: clocks. If each limiter instance uses its own clock to refill, skew between machines gives some callers extra tokens. Use Redis's TIME inside the script so one clock decides.

Interviewer pushback:

  • "Isn't the script a bottleneck, since it's single-threaded per shard?" → Each script is a few microseconds of work; at 50,000 a second per shard that's a fraction of a core. Spread keys across more shards if needed.

Memory code: "Read, decide and spend in one move, for every rule at once, on one clock."