Art of Codeartofcode
Lesson 5 of 16

6. Vertical vs Horizontal Scaling

1
The picture to keep in your head
2
6.1 What does "scaling" actually mean?
3
6.2 What is vertical scaling?
4
6.3 What is horizontal scaling?
5
6.4 Why does scaling out need stateless servers?
6
6.5 If servers are stateless, where does the state go?
7
6.6 What is autoscaling, and how do you get it right?
8
6.7 What about the database? Why is it the hardest part to scale?
9
6.8 What does each kind of scaling really cost?
10
6.9 So when do you choose which?
11
Memory card and where the analogy breaks
12
Practice: five levels
13
Interview questions
14
Solutions
15
Sources
16
Code lab
6. Vertical vs Horizontal Scaling

6.4 Why does scaling out need stateless servers?

3 min read+10 XP

In the kitchen: If the waiter tells one chef "table 4 wants the dosa without onion", only that chef knows. When table 4's next order goes to a different chef, it comes out wrong. Kitchens fix this with the order board: every order is written where any chef can read it, so any chef can cook any order.

In the system: A load balancer may send a user's next request to any server. That only works if no server keeps anything about the user in its own memory that the next request needs. Such servers are stateless (Chapter 3, section 3.10).

If servers are stateful (they keep sessions, carts or uploaded files locally):

  • Users get logged out or lose their cart when they land on another server (the Chapter 3 Level 4 bug).
  • You can't remove a server without losing what's on it, so scaling in, deploying and recovering from crashes all become painful.
  • New servers start empty and are useless to existing users.

Sticky sessions (the load balancer always sends a user to the same server) hide the problem, but load spreads unevenly and a dead server still loses its users' data. Treat them as a temporary patch (Chapter 7).

Memory code: "Any chef can cook any order." Then you can add or remove chefs freely.