Background Job Processing Systems Cheat Sheet

Updated 2026-05-28

Next Topic: Bun JavaScript Runtime Cheat Sheet

Background job processing systems enable applications to offload time-consuming tasks (email sending, image processing, data exports, API calls) to separate worker processes that execute asynchronously from the main request-response cycle. These systems prevent user-facing operations from blocking on long-running work by placing jobs into persistent queues backed by Redis, PostgreSQL, RabbitMQ, or cloud services like AWS SQS. Understanding job queue architectures, retry strategies, priority scheduling, and monitoring patterns is essential for building scalable, resilient systems that handle millions of tasks daily while gracefully managing failures. A well-designed background job system transforms a slow, brittle application into one that feels fast and handles spikes in load without degradation — every modern web application processing significant workload relies on one.

What This Cheat Sheet Covers

This topic spans 15 focused tables and 160 indexed concepts. Below is a complete table-by-table outline of this topic, spanning foundational concepts through advanced details.

Table 1: Job Queue Frameworks by LanguageTable 2: Message Brokers and Queue BackendsTable 3: Job Queue Architectural PatternsTable 4: Retry and Failure Handling StrategiesTable 5: Job Scheduling and TimingTable 6: Concurrency and Worker ManagementTable 7: Message Delivery Guarantees and SemanticsTable 8: Job Queue Monitoring and ObservabilityTable 9: Advanced Queue Features and PatternsTable 10: FIFO and Ordering GuaranteesTable 11: Job Queue Antipatterns and PitfallsTable 12: Cloud-Native Job ProcessingTable 13: Job Queue Performance OptimizationTable 14: Workflow Orchestration PlatformsTable 15: Real-World Use Cases

Table 1: Job Queue Frameworks by Language

The ecosystem of job queue libraries spans every major backend language; choosing the right one determines your operational model, performance ceiling, and infrastructure requirements. Most frameworks offer a spectrum from Redis-backed in-memory speed to database-backed ACID guarantees — with managed cloud platforms now offering a third path requiring no broker infrastructure at all.

Framework	Example	Description
Sidekiq	`class EmailJob` `include Sidekiq::Job` `def perform(user_id)` `User.find(user_id).send_welcome` `end` `end` `EmailJob.perform_async(123)`	• Ruby's most popular background job processor • multi-threaded (handles many jobs per process), uses Redis for storage, supports retries and scheduled jobs • powers Shopify, GitHub, and thousands of Rails apps.
BullMQ	`const queue = new Queue('emails');` `await queue.add('send', {` `userId: 123, type: 'welcome'` `});`	• Node.js and Bun job queue built on Redis with priorities, rate limiting, delayed jobs, parent-child workflows, and events • successor to Bull with better TypeScript support and performance.
Celery	`@app.task` `def send_email(user_id):` `user = User.objects.get(id=user_id)` `user.send_welcome()` `send_email.delay(123)`	• Python's distributed task queue standard • supports multiple brokers (RabbitMQ, Redis, SQS), result backends, periodic tasks with Celery Beat, and canvas workflows for complex orchestration.
Hangfire	`BackgroundJob.Enqueue(() =>` `SendEmail(123));` `RecurringJob.AddOrUpdate(` `"cleanup", () => Cleanup(),` `Cron.Daily);`	• .NET background job processor integrated with ASP.NET • uses SQL Server, PostgreSQL, or Redis for persistence, includes built-in dashboard, supports fire-and-forget, delayed, recurring, and continuation jobs.
Temporal	`@workflow.run` `async def order_workflow(order_id):` `await charge_payment(order_id)` `await ship_order(order_id)` `await send_notification(order_id)`	• Durable execution platform for workflows and long-running processes • uses event sourcing to capture state at every step, enabling automatic retries from exact failure point • handles multi-step sagas, human-in-the-loop, and distributed orchestration • supports Go, Java, Python, TypeScript, PHP.
Inngest	`inngest.createFunction(` `{ id: "send-email" },` `{ event: "user.created" },` `async ({ event }) => {` `await sendEmail(event.data);` `}` `);`	• Event-driven workflow platform for TypeScript/JavaScript • serverless-native with automatic retries, step functions, and durable execution • handles long-running workflows without managing infrastructure
Trigger.dev	`task("send-email", async (payload) => {` `await sendEmail(payload.userId);` `});`	• Open-source TypeScript background job platform with no timeouts — runs on long-lived compute, not serverless functions • automatic retries, real-time observability, elastic scaling, and support for AI/agent workflows • self-hostable or managed cloud.
Laravel Queue	`class SendEmailJob` `implements ShouldQueue {` `public function handle() {` `Mail::to($this->user)->send(...);` `}` `}` `SendEmailJob::dispatch($user);`	• PHP/Laravel's built-in async job system supporting Redis, SQS, database, and Beanstalkd drivers • job chaining, batching, rate limiting, and graceful failure handling built in • monitored via Laravel Horizon (Redis dashboard).
Active Job	`class EmailJob < ApplicationJob` `queue_as :emails` `def perform(user_id)` `User.find(user_id).send_email` `end` `end` `EmailJob.perform_later(123)`	• Rails' built-in abstraction layer for background jobs • doesn't process jobs itself but provides unified API for Sidekiq, Solid Queue, GoodJob, Resque, Delayed Job, and others • enables adapter-agnostic job code.
Asynq	`client := asynq.NewClient(...)` `task := asynq.NewTask("email:send",` `payload, asynq.MaxRetry(5))` `client.Enqueue(task)`	• Go task queue backed by Redis • supports weighted priority queues, strict priority queues, unique tasks, periodic tasks, task aggregation, and Prometheus metrics • includes Asynqmon web UI and CLI.

Table 1: Job Queue Frameworks by Language

Framework	Example	Description
Sidekiq	`class EmailJob` `include Sidekiq::Job` `def perform(user_id)` `User.find(user_id).send_welcome` `end` `end` `EmailJob.perform_async(123)`	• Ruby's most popular background job processor • multi-threaded (handles many jobs per process), uses Redis for storage, supports retries and scheduled jobs • powers Shopify, GitHub, and thousands of Rails apps.
BullMQ	`const queue = new Queue('emails');` `await queue.add('send', {` `userId: 123, type: 'welcome'` `});`	• Node.js and Bun job queue built on Redis with priorities, rate limiting, delayed jobs, parent-child workflows, and events • successor to Bull with better TypeScript support and performance.
Celery	`@app.task` `def send_email(user_id):` `user = User.objects.get(id=user_id)` `user.send_welcome()` `send_email.delay(123)`	• Python's distributed task queue standard • supports multiple brokers (RabbitMQ, Redis, SQS), result backends, periodic tasks with Celery Beat, and canvas workflows for complex orchestration.
Hangfire	`BackgroundJob.Enqueue(() =>` `SendEmail(123));` `RecurringJob.AddOrUpdate(` `"cleanup", () => Cleanup(),` `Cron.Daily);`	• .NET background job processor integrated with ASP.NET • uses SQL Server, PostgreSQL, or Redis for persistence, includes built-in dashboard, supports fire-and-forget, delayed, recurring, and continuation jobs.
Temporal	`@workflow.run` `async def order_workflow(order_id):` `await charge_payment(order_id)` `await ship_order(order_id)` `await send_notification(order_id)`	• Durable execution platform for workflows and long-running processes • uses event sourcing to capture state at every step, enabling automatic retries from exact failure point • handles multi-step sagas, human-in-the-loop, and distributed orchestration • supports Go, Java, Python, TypeScript, PHP.
Inngest	`inngest.createFunction(` `{ id: "send-email" },` `{ event: "user.created" },` `async ({ event }) => {` `await sendEmail(event.data);` `}` `);`	• Event-driven workflow platform for TypeScript/JavaScript • serverless-native with automatic retries, step functions, and durable execution • handles long-running workflows without managing infrastructure
Trigger.dev	`task("send-email", async (payload) => {` `await sendEmail(payload.userId);` `});`	• Open-source TypeScript background job platform with no timeouts — runs on long-lived compute, not serverless functions • automatic retries, real-time observability, elastic scaling, and support for AI/agent workflows • self-hostable or managed cloud.
Laravel Queue	`class SendEmailJob` `implements ShouldQueue {` `public function handle() {` `Mail::to($this->user)->send(...);` `}` `}` `SendEmailJob::dispatch($user);`	• PHP/Laravel's built-in async job system supporting Redis, SQS, database, and Beanstalkd drivers • job chaining, batching, rate limiting, and graceful failure handling built in • monitored via Laravel Horizon (Redis dashboard).
Active Job	`class EmailJob < ApplicationJob` `queue_as :emails` `def perform(user_id)` `User.find(user_id).send_email` `end` `end` `EmailJob.perform_later(123)`	• Rails' built-in abstraction layer for background jobs • doesn't process jobs itself but provides unified API for Sidekiq, Solid Queue, GoodJob, Resque, Delayed Job, and others • enables adapter-agnostic job code.
Asynq	`client := asynq.NewClient(...)` `task := asynq.NewTask("email:send",` `payload, asynq.MaxRetry(5))` `client.Enqueue(task)`	• Go task queue backed by Redis • supports weighted priority queues, strict priority queues, unique tasks, periodic tasks, task aggregation, and Prometheus metrics • includes Asynqmon web UI and CLI.