Message queues are asynchronous communication components that enable distributed systems to exchange data reliably without requiring direct connections between services. They act as buffers between producers and consumers, storing messages until they can be processed, which decouples system components and allows them to scale independently. Message queues solve critical problems in modern architectures: handling traffic spikes, enabling fault tolerance through retry mechanisms, and providing eventual consistency across distributed services. A key insight: Kafka 4.2 introduced Share Groups (KIP-932), bringing native queue semantics to streaming — blurring the historical line between message brokers (like RabbitMQ) and streaming platforms — but the core choice still hinges on whether you need per-message acknowledgment with task distribution or ordered, replayable event logs.
What This Cheat Sheet Covers
This topic spans 20 focused tables and 128 indexed concepts. Below is a complete table-by-table outline of this topic, spanning foundational concepts through advanced details.
Table 1: Core Concepts
| Concept | Example | Description |
|---|---|---|
producer.send(topic, msg) | • Component that creates and sends messages to a queue or topic • decoupled from consumers and does not wait for processing. | |
consumer.receive(queue) | • Component that retrieves and processes messages • can run independently of producers and scale separately. | |
orders_queue | • Ordered collection of messages following FIFO semantics • each message delivered to one consumer only. | |
user.events | Named channel for publishing messages that can be consumed by multiple independent subscribers simultaneously. | |
RabbitMQ, ActiveMQ | • Middleware that routes and delivers messages between producers and consumers • manages queues, exchanges, and delivery guarantees. | |
{id: "123", data: {...}} | • Unit of data containing headers (metadata) and payload (actual data) • immutable once published. |