Art of Codeartofcode
Lesson 16 of 17

7. Load Balancers, Reverse Proxies and API Gateways

1
The picture to keep in your head
2
7.1 Why do we need a load balancer?
3
7.2 What's the difference between a layer 4 and a layer 7 load balancer?
4
7.3 How does the load balancer choose a server?
5
7.4 How does the load balancer know a server is dead?
6
7.5 What are sticky sessions, and should you use them?
7
7.6 What is SSL termination?
8
7.7 If everything goes through the load balancer, what if it fails?
9
7.8 What's the difference between a forward proxy and a reverse proxy?
10
7.9 What is an API gateway?
11
7.10 How do you balance traffic across regions?
12
Memory card and where the analogy breaks
13
Practice: five levels
14
Interview questions
15
Solutions
16
Sources
17
Code lab
7. Load Balancers, Reverse Proxies and API Gateways

Sources

3 min read+10 XP

Source notes

  • Layer 4 and layer 7 refer to the transport and application layers of the OSI networking model.
  • Smooth weighted round robin as implemented here follows the algorithm used by NGINX's upstream module (described in NGINX's source and documentation).
  • "Pick two at random, use the less loaded" (the power of two random choices): Michael Mitzenmacher, "The Power of Two Choices in Randomized Load Balancing", IEEE Transactions on Parallel and Distributed Systems, 2001.
  • VRRP (Virtual Router Redundancy Protocol): RFC 5798. Keepalived is a common open-source implementation used for load balancer failover.
  • Health checks, thresholds and connection draining (deregistration delay) are described in cloud load balancer documentation (for example AWS Elastic Load Balancing target group health checks). Defaults vary by provider.
  • All bookstore traffic numbers are illustrative assumptions. Arithmetic computed 2026-10-08.
  • Level 3 and Level 4 code: tested 2026-10-08 (code/ch07_load_balancer.py, all checks pass).