23. Pub/Sub and Fan-Out
1
The picture to keep in your head2
23.1 What's the difference between a queue and pub/sub?3
23.2 What are topics and subscribers?4
23.3 What are consumer groups?5
23.4 What are the fan-out strategies?6
23.5 What is event-driven architecture?7
23.6 How do you handle duplicates and ordering?8
23.7 How do you publish events reliably (the outbox pattern)?9
Memory card and where the analogy breaks10
Practice: five levels11
Interview questions12
Solutions13
Sources14
Code lab23. Pub/Sub and Fan-Out
23.2 What are topics and subscribers?
3 min read+10 XP
On the paper round: The agency handles several titles: The Hindu, Sportstar, a Tamil daily. You subscribe to the ones you want, and the press for each title doesn't need to know who you are.
In the system:
- A topic is a named stream of related messages:
order_placed,book_released,payment_captured. - A publisher sends messages to a topic. It doesn't know who's listening, or how many.
- A subscription is a named interest in a topic. Each subscription gets every message published after it was created (and, with a log like Kafka, can also read older ones still retained).
Designing topics well:
- Name them after facts that happened, in the past tense:
order_placed, notsend_sms. - Include what consumers need (order ID, user, total, items), or at least the ID plus enough to decide whether to fetch more. Too thin, and every consumer calls back to the order service; too fat, and you leak internals.
- Version the schema (Chapter 24): add fields, don't rename or remove them, so old consumers keep working. Schema registries (Avro, Protobuf) help.
- One topic per kind of event, not one giant "everything" topic and not one per customer.
Memory code: "Name the topic after what happened; let anyone who cares subscribe."