25.4 How does distributed rate limiting with Redis work?

In production

At the club: The club has three doors. If each bouncer keeps their own count, a guest can use all three doors and get in three times as often. So all three bouncers share one guest list on a radio, and they must update it carefully so two of them don't each let in "the last guest".

In the system: Your API runs on many servers behind a load balancer (Chapter 7). A client's requests land on different servers, so each server's local count sees only part of the traffic. Options:

1. A shared counter in Redis (the usual answer):

• Fixed window: INCR rl:{key}:{window}, and set EXPIRE on first use. Compare the returned count with the limit.
• Token bucket or sliding window counter: a small Lua script that reads, updates and decides in one atomic step on the Redis server.
• Sliding log: a sorted set per key (ZREMRANGEBYSCORE to drop old entries, ZCARD to count, ZADD to record), inside a script or transaction.

Figure: count and check in one atomic step, or racing servers all see "under the limit".

The classic bug: GET the count, check it in the app, then SET count + 1. Under load, many servers read the same value and all decide they're under the limit. In the chapter's code, 20 concurrent requests arrive with 95 already used: read-then-write lets all 20 through; atomic INCR lets exactly the 5 remaining through. (This is the Level 4 bug, and the same race as Chapter 17's lost update.)

2. Local limits per server: give each of N servers limit ÷ N. No network call, but inaccurate when traffic isn't spread evenly. In the chapter's code, a client sending 100 requests a minute across 10 servers, each limited to 10, gets only 55 through because some servers got far more than their share. Fine as a rough backstop, not as the main limit.

3. Hybrid: count locally and sync with Redis every few hundred milliseconds, or take tokens from Redis in batches. Cheaper, slightly less exact.

Operational choices:

• Latency: a Redis round trip on every request (often under a millisecond in the same data centre). Keep Redis close to the API servers.
• Scale: at 1 million requests a minute, that's about 16,700 Redis operations a second; shard keys across a Redis cluster (Chapter 13).
• If Redis is down: fail open (allow requests, risking overload) or fail closed (reject, risking an outage). Most public APIs fail open with a local backstop limit; security limits like login attempts may fail closed.
• Clock skew: use Redis's time or window keys, not each server's clock.

Memory code: "One list for every door, updated in one move."

---