29.4 Why do you need timeouts?

Protecting calls

At home: If the kettle hasn't boiled in 5 minutes, something's wrong. You don't stand there all day.

In the system: Every network call needs a timeout: a maximum time to wait for a response. Without one, a slow dependency makes your service slow, then stuck: each waiting request holds a thread, a connection and memory, and they pile up until you run out (Chapter 27's Level 4).

Choosing a timeout:

• From measured latency: a little above the dependency's normal p99 or p99.9. In the chapter's code, a dependency with a median of about 89 ms has a p99 of about 231 ms and a p99.9 of about 312 ms; a timeout around 350 ms catches real trouble without cutting off normal slow requests.
• Within the caller's budget: if checkout must respond in 2 seconds, its downstream calls can't each take 2 seconds.
• Deadline propagation: pass the remaining time along ("you have 800 ms left") so deep calls don't keep working on requests the user has already abandoned (gRPC deadlines do this).
• Separate connect and read timeouts: connecting should be fast (tens of ms); reading may take longer.

Too short, and you fail requests that would have succeeded (and trigger retries). Too long, and slow dependencies pile up. Most library defaults are far too long, sometimes infinite.

Memory code: "Don't stand by the kettle all day."

---