27.5 What is the real cost of microservices?

In the restaurant: In a food court, a thali that needs rice from one stall, dal from another and dessert from a third means three queues. If one stall is closed, your thali is incomplete. And someone now has to manage the directory, the seating, the cleaning and the rules for every stall.

In the system: Every in-process call that becomes a network call brings the network's problems with it:

Figure: each hop adds latency and a way to fail.

• Latency adds up. An in-process call takes microseconds; a network call takes milliseconds. A request crossing 6 services at about 2 ms each spends 12 ms just travelling.
• Availability multiplies. If a request needs 6 services, each up 99.9% of the time, it succeeds only about 99.4% of the time: roughly 4.3 hours of failures a month instead of 43 minutes (computed in the chapter's code).
• Tail latency grows with fan-out. If a page calls 10 services in parallel and each is slow 1% of the time, about 10% of page loads wait for a slow one; at 100 calls, 63%.
• Partial failure becomes normal: you need timeouts, retries, circuit breakers and fallbacks (Chapter 29) on every call.
• No more simple transactions. Cross-service changes need sagas, outboxes and idempotency (Chapters 17, 23 and 24).
• Operational load: a deploy pipeline, monitoring, alerting, security and on-call rota per service; plus service discovery, an API gateway, distributed tracing (Chapter 30) and contract versioning.
• The distributed monolith: services that must all be deployed together, share a database, or call each other synchronously in long chains. All the costs of microservices, none of the benefits.

Even large companies hit this. In 2023 Amazon's Prime Video team wrote that its audio/video monitoring service, built as distributed serverless components, hit a scaling limit at around 5% of expected load and cost too much; moving it into a single process on containers cut its infrastructure cost by 90%.

Memory code: "Every stall you split off is another queue, another lock, another thing to run."

---