Apache Druid Real-Time Analytics Database Cheat Sheet_v1_tables

Apache Druid is a distributed, column-oriented, real-time analytics database purpose-built for sub-second OLAP queries on large-scale event-driven data. It sits at the intersection of data warehouses, timeseries databases, and search systems — combining columnar storage, bitmap indexes, and time-based partitioning with native streaming ingestion from Kafka and Kinesis. The core architectural insight that makes Druid fast is that data is always partitioned by time first, stored in immutable, pre-indexed segment files in deep storage, and optionally pre-aggregated using rollup — meaning query work is minimized before the query even arrives.

What This Cheat Sheet Covers

This topic spans 15 focused tables and 113 indexed concepts. Below is a complete table-by-table outline of this topic, spanning foundational concepts through advanced details.

Table 1: Core Architecture — Services and Their RolesTable 2: Druid Server Node Types (Deployment Grouping)Table 3: External DependenciesTable 4: Segments — Storage Unit and StructureTable 5: Time Partitioning and Secondary PartitioningTable 6: Ingestion MethodsTable 7: Kafka Supervisor Spec ConfigurationTable 8: Rollup and Aggregation StrategiesTable 9: Query Types — Native QueriesTable 10: Druid SQL and Query InterfaceTable 11: Deep Storage and Tiered StorageTable 12: Retention and Data Lifecycle ManagementTable 13: Query CachingTable 14: Query Performance TuningTable 15: Apache Druid vs. Apache Pinot vs. ClickHouse

Quick IndexSubscribe to unlock

A jump-to index of every table row in this cheat sheet.

Mind MapSubscribe to unlock

An interactive map of every table and concept in this topic.

Table 1: Core Architecture — Services and Their Roles

Every Druid service has a distinct, well-separated responsibility; understanding which service does what is the prerequisite for sizing, tuning, and troubleshooting any cluster.

Role	Example	Description
Broker	`druid.service=druid/broker`	Query gateway — receives queries from clients, fans them out to Historicals and MiddleManagers, then merges partial results and returns a single response.
Historical	`druid.service=druid/historical`	• Stores and serves immutable segments downloaded from deep storage • the primary query compute node for finalized data
Coordinator	`druid.service=druid/coordinator`	Segment lifecycle manager — balances segments across Historicals, enforces load/drop retention rules, and triggers compaction.
Overlord	`druid.service=druid/overlord`	Ingestion controller — accepts ingestion tasks, assigns them to MiddleManagers, creates task locks, and coordinates segment publishing.
MiddleManager	`druid.service=druid/middleManager`	Ingestion worker — spawns Peon child processes (one JVM per task) that read from sources and write new segments.
Peon	Spawned per ingestion task	• Single-task JVM child of a MiddleManager • executes exactly one ingestion task (streaming or batch) in isolation
Router	`druid.service=druid/router`	• Unified API gateway in front of Brokers, Overlord, and Coordinator • also hosts the web console UI • Routes queries to the appropriate Broker tier based on data retention rules
Indexer	`druid.service=druid/indexer`	• Alternative to MiddleManager + Peon — runs tasks as threads in a single JVM instead of forking per-task, enabling better resource sharing across streaming tasks • Currently experimental

Table 2: Druid Server Node Types (Deployment Grouping)

In practice, Druid services are grouped into three logical server types for deployment; knowing this grouping clarifies how to scale each dimension independently.

Node Type	Services Hosted	Description
Master server	Coordinator + Overlord	• Manages data availability and ingestion scheduling • for HA, run at least 2 Master servers

The rest of this section is part of the full cheat sheet

Table 3: External Dependencies

Druid relies on three external systems for coordination, metadata, and durable storage — these are not provided by Druid itself and must be provisioned separately.

Dependency	Example Config	Description
Deep storage	`druid.storage.type=s3`	• Durable object store (S3, GCS, Azure Blob, HDFS, or local) that holds all segment files permanently • Druid can fully rebuild from deep storage if all data nodes are lost

The rest of this section is part of the full cheat sheet

Table 4: Segments — Storage Unit and Structure

The segment is Druid's fundamental storage and indexing unit; almost every performance and operational concept in Druid ultimately comes back to segment properties.

Concept	Example	Description
Segment file	`metrics_2024-01-01/2024-01-02_v1_0`	• Immutable, columnar file containing data for one time interval (time chunk) • named `datasource_intervalStart_intervalEnd_version_partition`.

The rest of this section is part of the full cheat sheet

Table 1: Core Architecture — Services and Their Roles

Every Druid service has a distinct, well-separated responsibility; understanding which service does what is the prerequisite for sizing, tuning, and troubleshooting any cluster.

Role	Example	Description
Broker	`druid.service=druid/broker`	Query gateway — receives queries from clients, fans them out to Historicals and MiddleManagers, then merges partial results and returns a single response.
Historical	`druid.service=druid/historical`	• Stores and serves immutable segments downloaded from deep storage • the primary query compute node for finalized data
Coordinator	`druid.service=druid/coordinator`	Segment lifecycle manager — balances segments across Historicals, enforces load/drop retention rules, and triggers compaction.
Overlord	`druid.service=druid/overlord`	Ingestion controller — accepts ingestion tasks, assigns them to MiddleManagers, creates task locks, and coordinates segment publishing.
MiddleManager	`druid.service=druid/middleManager`	Ingestion worker — spawns Peon child processes (one JVM per task) that read from sources and write new segments.
Peon	Spawned per ingestion task	• Single-task JVM child of a MiddleManager • executes exactly one ingestion task (streaming or batch) in isolation
Router	`druid.service=druid/router`	• Unified API gateway in front of Brokers, Overlord, and Coordinator • also hosts the web console UI • Routes queries to the appropriate Broker tier based on data retention rules
Indexer	`druid.service=druid/indexer`	• Alternative to MiddleManager + Peon — runs tasks as threads in a single JVM instead of forking per-task, enabling better resource sharing across streaming tasks • Currently experimental

Table 2: Druid Server Node Types (Deployment Grouping)

In practice, Druid services are grouped into three logical server types for deployment; knowing this grouping clarifies how to scale each dimension independently.

Node Type	Services Hosted	Description
Master server	Coordinator + Overlord	• Manages data availability and ingestion scheduling • for HA, run at least 2 Master servers

Table 3: External Dependencies

Druid relies on three external systems for coordination, metadata, and durable storage — these are not provided by Druid itself and must be provisioned separately.

Dependency	Example Config	Description
Deep storage	`druid.storage.type=s3`	• Durable object store (S3, GCS, Azure Blob, HDFS, or local) that holds all segment files permanently • Druid can fully rebuild from deep storage if all data nodes are lost

Table 4: Segments — Storage Unit and Structure

The segment is Druid's fundamental storage and indexing unit; almost every performance and operational concept in Druid ultimately comes back to segment properties.

Concept	Example	Description
Segment file	`metrics_2024-01-01/2024-01-02_v1_0`	• Immutable, columnar file containing data for one time interval (time chunk) • named `datasource_intervalStart_intervalEnd_version_partition`.