Message queues are asynchronous communication components that enable distributed systems to exchange data reliably without requiring direct connections between services. They act as buffers between producers and consumers, storing messages until they can be processed, which decouples system components and allows them to scale independently. Message queues solve critical problems in modern architectures: handling traffic spikes, enabling fault tolerance through retry mechanisms, and providing eventual consistency across distributed services. A key insight: Kafka 4.2 introduced Share Groups (KIP-932), bringing native queue semantics to streaming — blurring the historical line between message brokers (like RabbitMQ) and streaming platforms — but the core choice still hinges on whether you need per-message acknowledgment with task distribution or ordered, replayable event logs.
What This Cheat Sheet Covers
This topic spans 20 focused tables and 128 indexed concepts, 122 flashcards. Below is a complete table-by-table outline of this topic, spanning foundational concepts through advanced details.
A jump-to index of every table row in this cheat sheet.
An interactive map of every table and concept in this topic.
Table 1: Core Concepts
The vocabulary here is what every other table builds on. Producers and consumers exchange messages through a broker, but the real subtlety lives in the distinction between a queue (one message, one consumer) and a topic (one message, many subscribers), and in Kafka-specific ideas like partitions and offsets that make ordered, replayable streams possible.
| Concept | Example | Description | |
|---|---|---|---|
producer.send(topic, msg) | • Component that creates and sends messages to a queue or topic • decoupled from consumers and does not wait for processing. | ||
consumer.receive(queue) | • Component that retrieves and processes messages • can run independently of producers and scale separately. | ||
orders_queue | • Ordered collection of messages following FIFO semantics • each message delivered to one consumer only. | ||
user.events | Named channel for publishing messages that can be consumed by multiple independent subscribers simultaneously. | ||
RabbitMQ, ActiveMQ | • Middleware that routes and delivers messages between producers and consumers • manages queues, exchanges, and delivery guarantees. | ||
{id: "123", data: {...}} | • Unit of data containing headers (metadata) and payload (actual data) • immutable once published. | ||
channel.ack(deliveryTag) | • Confirmation signal from consumer to broker that message was successfully processed • triggers removal from queue. | ||
order-processing-group | • Set of consumers sharing the same group ID that distribute partition workload • each partition consumed by one consumer in group. | ||
Topic split into 16 partitions | • Subdivision of a topic enabling parallel processing and horizontal scaling • each partition maintains an ordered sequence. | ||
Consumer at offset 1523 | • Position marker within a partition indicating which messages have been read or processed • enables replay and recovery. | ||
Exchange → Queue via key | • Rule that links an exchange to a queue using a routing key or pattern • determines how messages are routed to queues. |