Art of Codeartofcode
Lesson 2 of 17

7. Load Balancers, Reverse Proxies and API Gateways

1
The picture to keep in your head
2
7.1 Why do we need a load balancer?
3
7.2 What's the difference between a layer 4 and a layer 7 load balancer?
4
7.3 How does the load balancer choose a server?
5
7.4 How does the load balancer know a server is dead?
6
7.5 What are sticky sessions, and should you use them?
7
7.6 What is SSL termination?
8
7.7 If everything goes through the load balancer, what if it fails?
9
7.8 What's the difference between a forward proxy and a reverse proxy?
10
7.9 What is an API gateway?
11
7.10 How do you balance traffic across regions?
12
Memory card and where the analogy breaks
13
Practice: five levels
14
Interview questions
15
Solutions
16
Sources
17
Code lab
7. Load Balancers, Reverse Proxies and API Gateways

7.1 Why do we need a load balancer?

3 min read+10 XP

Load balancing

At the airport: Without a queue manager, three things go wrong. Some counters are swamped while others are empty. Passengers keep queueing at a counter that's broken. And you can't open or close counters without confusing everyone.

In the system: A load balancer sits between clients and a group of servers (often called a pool or target group). Every request reaches the load balancer first, and it forwards the request to one healthy server. It gives you:

  1. One address for many servers. DNS points shop.example.com at the load balancer (Chapter 4); the servers behind it can change freely.
  2. Even load. No server is swamped while others idle.
  3. Failure hiding. Dead servers are taken out of rotation automatically (7.4).
  4. Painless scaling and deploys. Add servers for the sale, remove them after, or replace them one at a time during a deploy, without users noticing (Chapter 6).

This only works if any server can handle any request, which is why servers should be stateless (Chapter 6, section 6.4).

Memory code: "One front door, many counters."