Art of Codeartofcode
Lesson 12 of 18

22. Message Queues

1
The picture to keep in your head
2
22.1 What are producers and consumers?
3
22.2 Why go async?
4
22.3 What are the delivery guarantees?
5
22.4 How do acks and visibility timeouts work?
6
22.5 What are retries and dead-letter queues?
7
22.6 What is an idempotent consumer?
8
22.7 How does ordering work?
9
22.8 What is backpressure?
10
22.9 How do you scale consumers?
11
22.10 Log (Kafka) vs queue (RabbitMQ, SQS): what's the difference?
12
22.11 How do delayed and scheduled jobs work?
13
Memory card and where the analogy breaks
14
Practice: five levels
15
Interview questions
16
Solutions
17
Sources
18
Code lab
22. Message Queues

22.11 How do delayed and scheduled jobs work?

3 min read+10 XP

At the bank: "Your new chequebook will be ready at 3 pm; come back then." The token isn't for now; it's for later.

In the system: Many jobs shouldn't run immediately:

  • "Cancel the order if it's still unpaid after 15 minutes."
  • "Send a payment reminder after 1 minute."
  • "Retry this failed SMS in 30 seconds."
  • "Every night at 2 am, rebuild the recommendations."

Ways to do it:

  • Delay on the message: SQS supports a delivery delay of up to 15 minutes; RabbitMQ has a delayed-message plugin or TTL plus dead-letter tricks.
  • A scheduler with a time-ordered store: keep jobs in a table or a sorted set (Redis ZADD with the run time as the score), and a poller moves due jobs onto the work queue. In the chapter's code, a heap-based scheduler releases the 1-minute reminder at second 60 and the 15-minute cancellation at second 900.
  • Cron-style schedules for recurring jobs, run by a scheduler that makes sure only one instance fires each run (a lock or leader, Chapter 17).

Make delayed jobs check before acting: "cancel if still unpaid" must re-read the order, because the customer may have paid in minute 14. And make them idempotent: the scheduler may fire twice.

Memory code: "A token for later: when it's called, check it still needs doing."