7.3 How does the load balancer choose a server?

At the airport: The queue manager can follow simple rules: take turns, send more people to the bigger counters, send people to the shortest queue, or always send the same family to the same counter.

In the system: These are the classic algorithms. Each is a different answer to "which server next?"

7.3.1 Round robin

At the airport: "Counter 1, counter 2, counter 3, counter 1, counter 2..." Everyone takes turns.

In the system: Servers get requests in a fixed rotation. Simple, cheap and fair when servers are identical and requests take about the same time. Its weakness: it ignores how busy each server is. If one server happens to get several slow requests (big searches, large uploads), it keeps getting its turn anyway and its queue grows.

7.3.2 Weighted round robin

At the airport: Counter 1 has two agents, so it gets two passengers for every one the others get.

In the system: Each server has a weight, usually matching its capacity. A server with weight 2 gets twice the requests of one with weight 1. Useful when servers differ in size (mixing old and new machines) or when you send a small share of traffic to a new version to test it. Good implementations spread the picks out ("big, small, big") instead of sending bursts ("big, big, ... small, small").

7.3.3 Least connections

At the airport: "Go to whichever counter has the shortest queue right now."

In the system: Each new request goes to the server with the fewest requests currently in progress. It adapts automatically when some requests are slow, because a server stuck on slow work has more open connections and gets fewer new ones.

Figure: with a mix of quick and slow requests, the chapter's simulation shows the busiest server peaking at 8 requests in progress under round robin and 5 under least connections.

Variants weigh in response time too ("least response time"), or pick two servers at random and use the less busy one, which works surprisingly well and avoids every load balancer rushing to the same "least busy" server at once.

7.3.4 IP hash

At the airport: "Families with a surname starting A to H always go to counter 1." The same passenger always lands at the same counter.

In the system: The load balancer hashes the client's IP address (or another key, like a user ID) and uses the result to pick a server, so the same client keeps reaching the same server. Useful when a server holds something per-user, like a local cache. Two catches:

• Many users behind one office or mobile network can share an IP, so load can be lumpy.
• With the simple version, hash(ip) mod number_of_servers, adding or removing one server reshuffles almost everyone (the Level 4 bug). Consistent hashing fixes that (Chapter 28).

Memory code: "Take turns, weigh the big ones, pick the shortest queue, or keep families together."

---