Practice: five levels

Assignments

Five levels. Do them in order. Answers are in "Solutions" at the end, so try first.

Level 1: Think (no code)

A. Sync or async? One line each.

1. Charging the customer's card during checkout.
2. Sending the order confirmation email.
3. Updating the search index after a product's price changes.
4. Checking stock before accepting an order.
5. Generating a monthly sales report.

B. Name the problem, or the fix.

1. A worker acks, then crashes before sending the SMS.
2. A customer gets two "order confirmed" SMS for one order.
3. One message with a malformed address is retried forever, and messages behind it wait.
4. "Shipped" is processed before "paid" for the same order.
5. A Kafka topic has 6 partitions; adding a 7th consumer to the group doesn't speed anything up.

Level 2: Calculate

1. With the figure's timings (save 60 ms, SMS 800 ms, slip 600 ms, points 300 ms), how long does checkout take synchronously? Asynchronously?
2. 1,200 messages a second arrive for 1 hour; workers handle 1,000 a second. How big is the backlog at the end? If arrivals then drop to 400 a second, how long to drain?
3. Messages arrive at 1,000 a second and each takes 200 ms. How many concurrent workers? With 50% headroom?
4. A message is retried with waits of 2, 4, 8, 16, 32 seconds before going to the DLQ. How long from the first failure to the DLQ (ignoring processing time)?
5. A Kafka topic has 12 partitions. What's the most consumers one group can use? Two different services each want every message: how many consumer groups?

Level 3: Build

In Python:

1. A Queue with send, receive(now), ack, a visibility timeout and a dead-letter queue after max_receives.
2. A deliver(mode, crash_points) that shows at-most-once losing messages and at-least-once repeating them.
3. A loyalty-points consumer with award_naive and award_idempotent, and a test that redelivers a message.
4. A Log with per-group offsets, poll(group) and seek(group, offset).
5. A Scheduler with schedule(run_at, job) and due(now).

Level 4: Debug

During the Big Sale, some customers got double loyalty points. Logs show the points worker sometimes takes 45 seconds per message (the points service is slow under load). The setup:

1. Explain exactly how one message produces two point awards.
2. Fix it in two ways: one about the timeout, one that makes duplicates harmless whatever the cause.

Level 5: Design

Design the background job system for the grocery app's checkout: after each order, send an SMS, print a packing slip at the right store, award loyalty points, and update analytics. Normal load is 200 orders a second; Big Sale peaks hit 1,200 for an hour. The SMS gateway allows 500 messages a second.

Deliver:

1. The queues or topics, and which consumers read which.
2. Delivery guarantee and idempotency for each consumer.
3. What happens during the Big Sale, given the SMS gateway limit.
4. Ordering needs (if any), and how you meet them.
5. Failure handling: retries, DLQs, alerts and redrive.