7. Load Balancers, Reverse Proxies and API Gateways
1
The picture to keep in your head2
7.1 Why do we need a load balancer?3
7.2 What's the difference between a layer 4 and a layer 7 load balancer?4
7.3 How does the load balancer choose a server?5
7.4 How does the load balancer know a server is dead?6
7.5 What are sticky sessions, and should you use them?7
7.6 What is SSL termination?8
7.7 If everything goes through the load balancer, what if it fails?9
7.8 What's the difference between a forward proxy and a reverse proxy?10
7.9 What is an API gateway?11
7.10 How do you balance traffic across regions?12
Memory card and where the analogy breaks13
Practice: five levels14
Interview questions15
Solutions16
Sources17
Code lab7. Load Balancers, Reverse Proxies and API Gateways
7.7 If everything goes through the load balancer, what if it fails?
3 min read+10 XP
At the airport: If there's only one queue manager and they faint, nobody reaches any counter, even though all the counters work. So airports keep a second manager standing by, ready to step in.
In the system: A single load balancer is a single point of failure (Chapter 29). Ways to make it redundant:
- Active-passive pair: two load balancers share one virtual IP address. The active one holds it; the standby watches with a heartbeat and claims the address within seconds if the active one stops answering (tools like keepalived use the VRRP protocol for this).
- Active-active: several load balancers all serve traffic, with DNS returning several addresses (Chapter 4) or the network spreading traffic across them (anycast, or equal-cost routing in the data centre).
- Managed cloud load balancers (like AWS Elastic Load Balancing or Google Cloud Load Balancing) are already spread across machines and zones by the provider. You don't manage the pair; you pay for the service.
Figure: the standby watches a heartbeat and claims the shared address if the active load balancer stops answering.
Also spread the servers behind it across availability zones (separate data centres in one region), so losing a whole building still leaves servers to balance across.
Memory code: "Every front door needs a spare door."