Art of Codeartofcode
Lesson 5 of 19

2. Back-of-Envelope Estimation

1
The picture to keep in your head
2
2.1 Why estimate at all?
3
2.2 Which units and sizes should you know?
4
2.3 What time shortcuts make the maths easy?
5
2.4 Which latency numbers should every developer know?
6
2.5 How do you turn daily users into requests per second?
7
2.6 Why does the read/write ratio matter?
8
2.7 Why size for the peak, not the average?
9
2.8 How do you estimate storage over the years?
10
2.9 How do you estimate bandwidth?
11
2.10 How much memory does the cache need?
12
2.11 How many servers do you need?
13
2.12 How do you round, and how do you know you're not way off?
14
Memory card and where the analogy breaks
15
Practice: five levels
16
Interview questions
17
Solutions
18
Sources
19
Code lab
2. Back-of-Envelope Estimation

2.4 Which latency numbers should every developer know?

3 min read+10 XP

At the wedding: Grabbing a ladle from the counter takes a second. Fetching rice from the store room takes minutes. Sending a van to the market takes an hour. You plan very differently for each.

In the system: Computers have the same ladder, and the gaps are enormous. Approximate figures (they vary by hardware and year, but the gaps between them hold):

OperationRough time
Read from memory (RAM)~100 nanoseconds
Random read from an SSD~150 microseconds
Round trip inside one data centre~500 microseconds
Disk seek on a spinning hard disk~10 milliseconds
Round trip from California to the Netherlands and back~150 milliseconds

1 millisecond = 1,000 microseconds = 1,000,000 nanoseconds.

if reading memory took 1 second, a disk seek would take 28 hours.

Figure: scale memory up to 1 second, and a trip across continents takes 17 days.

What it means for design:

  • Memory is far faster than anything over a network or on disk. That's why caches exist (Chapter 12).
  • Every network hop costs real time. A request that makes 10 calls in a row between services pays that cost 10 times.
  • Distance can't be optimised away. Light in fibre takes time to cross an ocean. That's why CDNs put copies near users (Chapter 26).

Memory code: "Nano for memory, micro for the building, milli for the disk and the world."