Cloud Message Queues and Event-Driven Architecture Cheat Sheet

Updated 2026-05-25

🧠Study flashcards on this topic136 cards · spaced repetition→

Cloud message queues and event-driven architectures enable asynchronous communication between distributed services by decoupling producers and consumers through message brokers. These systems form the backbone of modern microservices, supporting patterns from simple point-to-point messaging to event streaming with exactly-once delivery guarantees and sophisticated routing. Kafka's evolution to KRaft mode in version 4.0 (March 2025) — removing ZooKeeper entirely — alongside the CloudEvents CNCF standard and a growing ecosystem of alternatives (Pulsar, NATS JetStream, Redpanda) makes broker selection and schema management more impactful than ever. Understanding tradeoffs between delivery models, queue types, and messaging patterns is essential for building scalable, resilient distributed systems that handle failures gracefully while maintaining data consistency.

What This Cheat Sheet Covers

This topic spans 16 focused tables and 160 indexed concepts, 136 flashcards. Below is a complete table-by-table outline of this topic, spanning foundational concepts through advanced details.

Table 1: Major Cloud Message Queue and Streaming ServicesTable 2: Queue Types and Delivery ModesTable 3: Delivery GuaranteesTable 4: Messaging PatternsTable 5: Event-Driven Architecture (EDA) PatternsTable 6: Message Ordering and PartitioningTable 7: Message Filtering and RoutingTable 8: Flow Control and BackpressureTable 9: Retry and Error HandlingTable 10: Message Durability and PersistenceTable 11: Performance and ScalabilityTable 12: Security and AuthenticationTable 13: Monitoring and ObservabilityTable 14: Testing and Development PatternsTable 15: Schema Management and Event ContractsTable 16: CDC, Kafka Connect, and Streaming Integration

Quick IndexSubscribe to unlock

A jump-to index of every table row in this cheat sheet.

Mind MapSubscribe to unlock

An interactive map of every table and concept in this topic.

Table 1: Major Cloud Message Queue and Streaming Services

The broker or service you choose defines your latency floor, throughput ceiling, operational model, and available delivery semantics. Compare services by their use-case fit — ordered streaming, fan-out notifications, managed queues, or lightweight edge messaging — before optimizing configuration.

Service	Example	Description
Apache Kafka	`kafka-topics.sh --create --topic orders --partitions 12`	• Distributed append-only log • high-throughput ordered streaming • as of Kafka 4.0 (Mar 2025) ZooKeeper fully removed — KRaft mode only • current stable: 4.3 (May 2026)
Amazon SQS	`aws sqs send-message --queue-url $URL --message-body "order-123"`	• Fully managed queue • Standard (at-least-once, best-effort order) or FIFO (exactly-once, strict order) • up to 256 KB per message • serverless scaling
Amazon SNS	`aws sns publish --topic-arn $ARN --message "event"`	• Managed pub/sub fanout • pushes to SQS, Lambda, HTTP, email, SMS • FIFO topics preserve order + deduplication
Amazon Kinesis Data Streams	`PutRecords(StreamName="clickstream", Records=[...])`	• Shard-based streaming for high-volume real-time data (clickstream, logs, telemetry) • multiple independent consumers • 24 h–365 d retention • ~200 ms latency
Amazon MSK	`aws kafka create-cluster --cluster-name prod-kafka`	• Fully managed Kafka on AWS • supports Kafka 4.1+ (Oct 2025), KRaft mode • MSK Express Brokers (2024+): 3× throughput, 20× faster scaling • MSK Serverless, MSK Connect, MSK Replicator for cross-region
Azure Service Bus	`sender.send_messages(ServiceBusMessage("payload"))`	• Enterprise queue/topic with sessions, transactions, dead-letter queues • Queues (point-to-point) or Topics+Subscriptions (pub/sub) • 256 KB–100 MB messages
Azure Event Hubs	`producer.send_batch(event_data_batch)`	• Kafka-compatible managed streaming • partitioned consumer groups • Capture writes to Blob/ADLS • up to 90-day retention
Google Pub/Sub	`publisher.publish(topic_path, data=b"event")`	• Serverless global pub/sub • push or pull delivery • exactly-once processing with subscriptions • 7-day default / up to 31-day retention • dead-letter topics
RabbitMQ	`channel.basic_publish(exchange="orders", routing_key="new", body=msg)`	• AMQP broker with flexible routing via exchanges (direct/fanout/topic/headers) • Quorum Queues (default since 3.x) for HA • Streams plugin for persistent log access
Apache Pulsar	`producer = client.create_producer("persistent://tenant/ns/topic")`	• Cloud-native messaging with stateless brokers + BookKeeper storage • native multi-tenancy, built-in geo-replication, 4 subscription types • tiered storage to S3/GCS • built-in schema registry
NATS / NATS JetStream	`js.publish("orders.new", b"payload")`	• CNCF lightweight broker • sub-millisecond latency • single binary with pub/sub + JetStream streaming + KV store + object store • 45+ client libs • ideal for IoT and edge
Redpanda	`rpk topic create orders --partitions 12`	• Kafka API-compatible C++ platform • no JVM, no ZooKeeper • 10× lower tail latency • built-in schema registry and Kafka Connect • 3–6× cheaper operational cost than self-hosted Kafka
AWS EventBridge	`events.put_events(Entries=[{"Source":"app","DetailType":"order","Detail":"{...}"}])`	• Serverless event bus routing AWS service events, SaaS events, and custom events via JSON rules to 20+ targets • schema registry • replays • event archive
EventBridge Pipes	Source: SQS → Filter → Enrich (Lambda) → Target: Step Functions	• Point-to-point integration with optional filter and enrichment steps • connects SQS, DynamoDB Streams, Kinesis, MSK, MQ to 15+ target services • eliminates glue-code Lambda functions

Table 2: Queue Types and Delivery Modes

Each queue type trades ordering guarantees for throughput or vice versa; choosing wrong early forces costly migrations later.

Type	Example	Description
Standard Queue	SQS Standard, Kafka default topic	• At-least-once delivery • best-effort ordering • near-unlimited throughput • consumers must be idempotent

The rest of this section is part of the full cheat sheet

Table 3: Delivery Guarantees

Delivery guarantees are a contract between the broker and consumer; breaking that contract silently is one of the most common causes of data loss or duplication in distributed systems.

Guarantee	Example	Description
At-Most-Once	Fire-and-forget; ack before processing	• Message may be lost but never duplicated • lowest overhead • acceptable for logs, metrics, ephemeral notifications

The rest of this section is part of the full cheat sheet

Table 4: Messaging Patterns

Messaging patterns are higher-level compositions that determine how producers, brokers, and consumers collaborate; picking the right pattern often matters more than broker configuration.

Pattern	Example	Description
Publish/Subscribe (Pub/Sub)	SNS topic → multiple SQS queues	• One producer • multiple independent consumers each receive a copy • decouples producers from consumers entirely

The rest of this section is part of the full cheat sheet

Table 1: Major Cloud Message Queue and Streaming Services

Service	Example	Description
Apache Kafka	`kafka-topics.sh --create --topic orders --partitions 12`	• Distributed append-only log • high-throughput ordered streaming • as of Kafka 4.0 (Mar 2025) ZooKeeper fully removed — KRaft mode only • current stable: 4.3 (May 2026)
Amazon SQS	`aws sqs send-message --queue-url $URL --message-body "order-123"`	• Fully managed queue • Standard (at-least-once, best-effort order) or FIFO (exactly-once, strict order) • up to 256 KB per message • serverless scaling
Amazon SNS	`aws sns publish --topic-arn $ARN --message "event"`	• Managed pub/sub fanout • pushes to SQS, Lambda, HTTP, email, SMS • FIFO topics preserve order + deduplication
Amazon Kinesis Data Streams	`PutRecords(StreamName="clickstream", Records=[...])`	• Shard-based streaming for high-volume real-time data (clickstream, logs, telemetry) • multiple independent consumers • 24 h–365 d retention • ~200 ms latency
Amazon MSK	`aws kafka create-cluster --cluster-name prod-kafka`	• Fully managed Kafka on AWS • supports Kafka 4.1+ (Oct 2025), KRaft mode • MSK Express Brokers (2024+): 3× throughput, 20× faster scaling • MSK Serverless, MSK Connect, MSK Replicator for cross-region
Azure Service Bus	`sender.send_messages(ServiceBusMessage("payload"))`	• Enterprise queue/topic with sessions, transactions, dead-letter queues • Queues (point-to-point) or Topics+Subscriptions (pub/sub) • 256 KB–100 MB messages
Azure Event Hubs	`producer.send_batch(event_data_batch)`	• Kafka-compatible managed streaming • partitioned consumer groups • Capture writes to Blob/ADLS • up to 90-day retention
Google Pub/Sub	`publisher.publish(topic_path, data=b"event")`	• Serverless global pub/sub • push or pull delivery • exactly-once processing with subscriptions • 7-day default / up to 31-day retention • dead-letter topics
RabbitMQ	`channel.basic_publish(exchange="orders", routing_key="new", body=msg)`	• AMQP broker with flexible routing via exchanges (direct/fanout/topic/headers) • Quorum Queues (default since 3.x) for HA • Streams plugin for persistent log access
Apache Pulsar	`producer = client.create_producer("persistent://tenant/ns/topic")`	• Cloud-native messaging with stateless brokers + BookKeeper storage • native multi-tenancy, built-in geo-replication, 4 subscription types • tiered storage to S3/GCS • built-in schema registry
NATS / NATS JetStream	`js.publish("orders.new", b"payload")`	• CNCF lightweight broker • sub-millisecond latency • single binary with pub/sub + JetStream streaming + KV store + object store • 45+ client libs • ideal for IoT and edge
Redpanda	`rpk topic create orders --partitions 12`	• Kafka API-compatible C++ platform • no JVM, no ZooKeeper • 10× lower tail latency • built-in schema registry and Kafka Connect • 3–6× cheaper operational cost than self-hosted Kafka
AWS EventBridge	`events.put_events(Entries=[{"Source":"app","DetailType":"order","Detail":"{...}"}])`	• Serverless event bus routing AWS service events, SaaS events, and custom events via JSON rules to 20+ targets • schema registry • replays • event archive
EventBridge Pipes	Source: SQS → Filter → Enrich (Lambda) → Target: Step Functions	• Point-to-point integration with optional filter and enrichment steps • connects SQS, DynamoDB Streams, Kinesis, MSK, MQ to 15+ target services • eliminates glue-code Lambda functions

Table 2: Queue Types and Delivery Modes

Each queue type trades ordering guarantees for throughput or vice versa; choosing wrong early forces costly migrations later.

Type	Example	Description
Standard Queue	SQS Standard, Kafka default topic	• At-least-once delivery • best-effort ordering • near-unlimited throughput • consumers must be idempotent

Table 3: Delivery Guarantees

Delivery guarantees are a contract between the broker and consumer; breaking that contract silently is one of the most common causes of data loss or duplication in distributed systems.

Guarantee	Example	Description
At-Most-Once	Fire-and-forget; ack before processing	• Message may be lost but never duplicated • lowest overhead • acceptable for logs, metrics, ephemeral notifications

Table 4: Messaging Patterns

Messaging patterns are higher-level compositions that determine how producers, brokers, and consumers collaborate; picking the right pattern often matters more than broker configuration.

Pattern	Example	Description
Publish/Subscribe (Pub/Sub)	SNS topic → multiple SQS queues	• One producer • multiple independent consumers each receive a copy • decouples producers from consumers entirely