23. Pub/Sub and Fan-Out
The picture to keep in your head
Every question below is answered twice: first on the paper round, so you can picture it, then in the system, so you know exactly how it works. Each answer ends with a memory code to lock it in.
The picture to keep in your head
Every morning, the newspaper is printed once at the press. Then every subscriber gets their own copy. The press doesn't phone each reader, and doesn't care how many there are; the paper agency keeps the list. Delivery boys split the area between them, so each house gets one paper, not five. Most papers are delivered to the doorstep, but a few readers prefer to collect from the stall when they want it. And the agency keeps an order book, so a new subscription is never forgotten even if the phone line dropped while you were signing up.
That's publish/subscribe: one side announces "this happened", and everyone who signed up gets a copy.
Our running example is the bookstore again, in two places:
- The "order placed" event, which the SMS, loyalty and analytics services all need to hear about.
- The "follow an author" feature: when an author releases a book, every follower should see it under "New from authors you follow". Most authors have a few hundred followers; a handful of famous ones have millions (numbers are assumptions for practice).
| On the paper round | In the system |
|---|---|
| The press prints once | A publisher publishes an event once |
| Every subscriber gets their own copy | Pub/sub: each subscription receives every message |
| The newspaper title | A topic |
| Delivery boys sharing one area | A consumer group sharing a subscription |
| Delivering to every doorstep | Fan-out on write (push) |
| Collecting from the stall when you want it | Fan-out on read (pull) |
| Doorstep for most, stall for the giant bundles | The hybrid fix for the celebrity problem |
| The agency's order book | The outbox pattern |
Figure: a queue hands a job to one worker; pub/sub hands the news to everyone who signed up.