5.6 What is Little's law?

On the highway: If 6,000 cars enter the highway every hour and each spends 2 hours on it, then at any moment there are 12,000 cars on the road. You didn't need to count them: arrivals per hour × hours each one stays.

In the system: Little's law says that, in a steady system:

requests in the system = arrival rate × time each request spends inside L = λ × W

It holds for almost any stable system, whatever happens inside, which makes it one of the most useful formulas in system design. (It's named after John Little, who proved it in 1961.)

What it's good for:

• Sizing concurrency. The bookstore gets 5,000 requests a second and each takes 200 ms (0.2 s). So about 5,000 × 0.2 = 1,000 requests are in progress at any moment. You need enough worker threads, database connections and memory for about 1,000 at once, or they queue.
• Seeing why slowness snowballs. If a slow database pushes latency from 200 ms to 2 seconds at the same traffic, requests in flight jump from 1,000 to 10,000. Thread pools and connection pools run out, new requests wait in queues, and latency grows further.
• Working backwards. If a pool holds at most 100 connections and each query takes 50 ms, the pool can complete at most 100 ÷ 0.05 = 2,000 queries a second.

Memory code: "Cars on the road = cars per hour × hours on the road." L = λ × W.

---