Art of Codeartofcode
Lesson 2 of 18

22. Message Queues

1
The picture to keep in your head
2
22.1 What are producers and consumers?
3
22.2 Why go async?
4
22.3 What are the delivery guarantees?
5
22.4 How do acks and visibility timeouts work?
6
22.5 What are retries and dead-letter queues?
7
22.6 What is an idempotent consumer?
8
22.7 How does ordering work?
9
22.8 What is backpressure?
10
22.9 How do you scale consumers?
11
22.10 Log (Kafka) vs queue (RabbitMQ, SQS): what's the difference?
12
22.11 How do delayed and scheduled jobs work?
13
Memory card and where the analogy breaks
14
Practice: five levels
15
Interview questions
16
Solutions
17
Sources
18
Code lab
22. Message Queues

22.1 What are producers and consumers?

3 min read+10 XP

The basics

At the bank: You (the customer) create the work by taking a token. The clerks do the work. You never talk to a specific clerk, and they never need to know who's coming next. The board sits between you.

In the system:

  • A producer sends messages to a queue: "order 881 placed, send the SMS".
  • A consumer (also called a worker) receives messages from the queue, does the work, and acknowledges each one when it's done.
  • The broker is the queue software: Amazon SQS, RabbitMQ, Apache Kafka, Google Pub/Sub, Redis streams and others.
python

The key property is decoupling: the producer doesn't know or care who handles the message, how many workers there are, or whether they're running right now. The queue holds messages until someone is ready.

Memory code: "Leave the job on the board; whoever's free picks it up."