Art of Codeartofcode
Lesson 8 of 15

32. Design a Rate Limiter as a Service

1
The interview prompt
2
32.1 What do you ask first?
3
32.2 How big is this, really?
4
32.3 What are the doors?
5
32.4 What do we store?
6
32.5 What does the whole system look like?
7
32.6 How do you pick the algorithm?
8
32.7 How do counters work in Redis?
9
32.8 What race conditions do you have to handle?
10
32.9 How do limits work across regions?
11
32.10 What would you say in the last five minutes?
12
Score yourself
13
Variations to practise
14
Sources
15
Code lab
32. Design a Rate Limiter as a Service

32.7 How do counters work in Redis?

3 min read+10 XP

One script per check. A Lua script runs atomically on the Redis server: no other command runs in the middle. The token-bucket script:

lua

Details that matter:

  • One round trip per check (32.2), and the script is loaded once and called by hash (EVALSHA).
  • TTL on every key, so 10 million idle keys don't live forever.
  • Shard by key across a Redis Cluster; one key's state lives on one shard.
  • Replicas for failover. Losing a shard briefly resets some buckets to full, which over-allows for a moment; acceptable for API limits, and one reason security rules keep a strict local fallback.
  • Hot keys: one giant tenant hammering one key concentrates load on one shard. Give very large tenants local token batches: the client takes, say, 50 tokens at once and spends them locally, calling Redis 50× less often, at the cost of slight inaccuracy.

Memory code: "One script, Redis's clock, a TTL on every key."