7. Load Balancers, Reverse Proxies and API Gateways
The picture to keep in your head
Every question below is answered twice: first at the airport, so you can picture it, then in the system, so you know exactly how it works. Each answer ends with a memory code to lock it in.
The picture to keep in your head
At a busy airport, passengers don't pick a check-in counter themselves. If they did, everyone would crowd the first counter while others sat empty. Instead, a queue manager stands at the front and says "counter 4, please". They send you to a counter that's free, skip the counter whose computer just crashed, and open more counters when a big flight arrives. Further in, a security gate checks everyone's ID before they reach the gates, and a desk at the terminal entrance sends you to the right airline.
That's a load balancer: one front door that hands each request to one of many servers. The security gate and the airline desk are its bigger cousins, the API gateway and layer-7 routing.
Our running example: the bookstore from Chapter 6, now running 68 identical app servers for the Book Fest sale. Users still type one address, shop.example.com. Something has to spread their requests across all 68.
| At the airport | In the system |
|---|---|
| Passengers arriving all day | Requests from users |
| The queue manager at the front | The load balancer |
| Check-in counters | App servers |
| A "closed" sign on a broken counter | Health checks removing a dead server |
| "Counter 4 is free, go there" | The balancing algorithm |
| The security gate checking IDs | API gateway (auth, rate limits) |
| The desk that sends you to your airline | Layer-7 routing by path or header |
Figure: the airport, mapped to the system. Passengers never pick their own counter; one person at the front decides.