Art of Codeartofcode
Lesson 4 of 16

6. Vertical vs Horizontal Scaling

1
The picture to keep in your head
2
6.1 What does "scaling" actually mean?
3
6.2 What is vertical scaling?
4
6.3 What is horizontal scaling?
5
6.4 Why does scaling out need stateless servers?
6
6.5 If servers are stateless, where does the state go?
7
6.6 What is autoscaling, and how do you get it right?
8
6.7 What about the database? Why is it the hardest part to scale?
9
6.8 What does each kind of scaling really cost?
10
6.9 So when do you choose which?
11
Memory card and where the analogy breaks
12
Practice: five levels
13
Interview questions
14
Solutions
15
Sources
16
Code lab
6. Vertical vs Horizontal Scaling

6.3 What is horizontal scaling?

3 min read+10 XP

In the kitchen: Hire more chefs who cook the same menu side by side. Ten chefs can cook roughly ten times the plates, and if one is sick, the other nine keep going. But it only works if the work can be split: if every plate needs the one chef who makes the secret sauce, the others stand around waiting.

In the system: Horizontal scaling (scaling out) means adding more machines that do the same job, with a load balancer spreading requests across them (Chapter 7).

scale up vs scale out, with the good and bad of each.

Figure: scaling up is simple but has a ceiling; scaling out has almost no ceiling but needs stateless servers and a load balancer.

Why it's the default for big systems:

  • Almost no ceiling: need more capacity, add more servers.
  • Survives failures: one server dies, the others carry on.
  • No downtime to grow: add servers while the old ones keep serving.
  • Cheap, ordinary machines, and you can remove them when traffic drops (6.6).

The catch: the work must split. The speed-up you get is limited by the part of the work that can't be spread out: a shared lock, a single counter every request updates, one database row everybody writes to. This is Amdahl's law:

best speed-up with n servers = 1 / (serial part + (1 - serial part) / n)

speed-up with 1 to 1,000 servers when 0%, 1% and 10% of the work can't be split.

Figure: if 10% of the work can't be split, even 1,000 servers give less than 10 times the speed.

If 10% of each request waits on one shared thing, no number of servers makes it more than 10 times faster. That's why scaling out goes hand in hand with removing shared bottlenecks.

Memory code: "More chefs, but only if the work can be split."