Lesson 10 of 14
23. Pub/Sub and Fan-Out

Practice: five levels

3 min read+10 XP

Assignments

Five levels. Do them in order. Answers are in "Solutions" at the end, so try first.

Level 1: Think (no code)

A. Queue or topic? One line each.

  1. "Resize this uploaded book cover."
  2. "Order 881 was placed."
  3. "Book 42's price changed."
  4. "Send this password-reset email."
  5. "Author 7 released a new book."

B. True or false? Explain in one line.

  1. In pub/sub, two instances of the same service in one consumer group both process every message.
  2. Fan-out on write makes reading a feed cheap.
  3. Fan-out on read is expensive for celebrities who post.
  4. "Save the order, then publish the event" can never lose an event.
  5. With the outbox pattern, subscribers may still receive duplicates.

Level 2: Calculate

  1. A celebrity author has 5 million followers. With fan-out on write at 50,000 feed writes a second, how long until the last follower's feed has the new book?
  2. 10,000 new books are released a day, by authors with 200 followers on average. How many feed writes a day with fan-out on write?
  3. 1 million feed loads a day, and each user follows 300 authors on average. How many author lookups a day with fan-out on read?
  4. 1,000 orders a second publish order_placed, and 4 services subscribe. How many deliveries a second does the broker make?
  5. The outbox relay polls every 1 second and publishing takes about 20 ms. What's the worst-case delay between commit and publish?

Level 3: Build

In Python:

  1. A Broker with subscribe(topic, group, member) and publish(topic, msg): every group gets every message; members inside a group share them.
  2. FanoutOnWrite, FanoutOnRead and Hybrid feed classes with post and read. Show they return identical feeds and compare their write and read counts.
  3. A StockView consumer that ignores duplicates by event ID and stale events by version.
  4. An outbox: write the order and the outbox row together (all or nothing), and a relay that may publish twice after a crash.
  5. Reproduce both dual-write failures (ghost event, lost event).

Level 4: Debug

Customers report SMS messages saying "Your order 883 is confirmed!" for orders that don't appear in their account. The checkout code:

python
  1. Explain exactly how a customer gets an SMS for an order that doesn't exist.
  2. Moving the publish after the transaction "fixes" that. What new problem does it create?
  3. Fix it properly.

Level 5: Design

Design the bookstore's "New from authors you follow" feature: 20 million readers, 200,000 authors, most with under 1,000 followers but a few with 5 million; readers open the feed about 5 times a day; about 10,000 books are released a day.

Deliver:

  1. How a release reaches followers (push, pull, or hybrid, and the threshold).
  2. Where feeds are stored and how big they get.
  3. How the release event is published reliably from the catalogue service.
  4. How duplicates and late events are handled.
  5. What other services subscribe to book_released, and how they're kept independent.