25. Rate Limiting
The picture to keep in your head
Every question below is answered twice: first at the club, so you can picture it, then in the system, so you know exactly how it works. Each answer ends with a memory code to lock it in.
The picture to keep in your head
A popular nightclub has a bouncer and a bowl of entry passes. Every few seconds, a staff member drops one new pass into the bowl, up to a maximum of 100. Each guest takes a pass to get in. If the bowl has passes, you walk straight in, even with a group of friends. If it's empty, the bouncer says "full, wait a minute", and points at the clock: "the next pass drops in 6 seconds".
The club doesn't do this to be rude. It does it so the room never gets so packed that nobody has a good time, so one rowdy group can't take every spot, and so the fire marshal stays happy. The bouncer also keeps an eye out for one gang trying to get in a hundred times with fake IDs.
That's rate limiting: let everyone in at a pace the system can handle, and say clearly when to come back.
Our running example: the bookstore's partner API, which allows 100 requests a minute per API key on the free plan, and its login endpoint, which allows 5 attempts per 15 minutes per account (limits are assumptions for practice).
| At the club | In the system |
|---|---|
| Keeping the room from overflowing | Why limit: protection, fairness, cost |
| A bowl of passes, refilled steadily | Token bucket |
| A queue that lets one in every few seconds | Leaky bucket |
| A clicker that resets on the hour | Fixed window |
| A guest book with every entry time | Sliding log |
| "Last hour's count, partly, plus this hour's" | Sliding window counter |
| One guest list shared by every door | Distributed rate limiting with Redis |
| Bouncer at the gate, at the bar, at the VIP room | Where to put the limiter |
| "Full: come back in 6 seconds" | 429 and rate-limit headers |
| One gang with many fake IDs | Bots and abuse |
Figure: let everyone in at a pace the room can handle, and say clearly when to come back.