Delta Lake Cheat Sheet

Updated 2026-04-20

🧠Study flashcards on this topic108 cards · spaced repetition→

Delta Lake is an open-source storage framework that brings ACID transactions, scalable metadata handling, and time travel to cloud data lakes. Built on top of Parquet, it provides a transactional layer through an append-only commit log (_delta_log) that records every change, enabling reliable concurrent writes and schema evolution without sacrificing performance. Originally developed by Databricks and now a Linux Foundation project, Delta Lake has reached version 4.2.0 (on Apache Spark 4.1.0) and serves as the foundation for modern lakehouse architectures across AWS S3, Azure ADLS, and Google Cloud Storage.

What This Cheat Sheet Covers

This topic spans 17 focused tables and 129 indexed concepts, 108 flashcards. Below is a complete table-by-table outline of this topic, spanning foundational concepts through advanced details.

Table 1: Core ConceptsTable 2: Read and Write OperationsTable 3: Table ManipulationTable 4: Schema ManagementTable 5: Time Travel and VersioningTable 6: OptimizationTable 7: Change Data CaptureTable 8: Concurrency and IsolationTable 9: Table CloningTable 10: ConstraintsTable 11: Advanced FeaturesTable 12: Partitioning StrategiesTable 13: Storage ConfigurationTable 14: InteroperabilityTable 15: Table PropertiesTable 16: SQL DDL CommandsTable 17: Delta Lake vs Alternatives

Quick IndexSubscribe to unlock

A jump-to index of every table row in this cheat sheet.

Mind MapSubscribe to unlock

An interactive map of every table and concept in this topic.

Table 1: Core Concepts

Everything else in Delta Lake rests on the handful of building blocks here — the transaction log that records every change, the Parquet files holding the actual data, and the protocol versions and table features that decide which clients can read or write a table. Once you see how the log, checkpoints, and the lakehouse model fit together, the rest of the cheat sheet reads far more naturally.

Concept	Example	Description
Transaction log	`_delta_log/00000000000000000000.json`	• Append-only JSON log that records every table change • each commit creates a new log file numbered sequentially, enabling ACID guarantees and time travel
ACID transactions	Multiple writers commit simultaneously	• Atomicity, Consistency, Isolation, Durability via optimistic concurrency control • failed transactions roll back without affecting committed data
Parquet data files	`part-00000-<uuid>.snappy.parquet`	• Columnar storage format containing actual data • Delta adds metadata layer on top for transactions and versioning
Checkpoint	`_delta_log/00000000000000000010.checkpoint.parquet`	• Parquet snapshot of table state written every 10 commits (default) • accelerates metadata reads by avoiding replay of thousands of JSON log entries • v2 checkpoints available via `delta.checkpointPolicy = 'v2'`
Table protocol version	`minReaderVersion=3, minWriterVersion=7`	• Protocol defines minimum client capabilities required to read/write a table • higher versions unlock features; supports table features model for granular opt-in
Table features	`deletionVectors`, `columnMapping`, `v2Checkpoint`	• Opt-in capabilities that enhance table functionality • enabling a feature upgrades table protocol and may restrict older client access
Lakehouse architecture	Delta tables + compute engines	• Storage layer supporting both BI and ML workloads on a single copy of data • combines warehouse reliability with lake scale and cost
Catalog-managed tables	`delta.feature.catalogManaged = 'supported'`	• Shifts commit coordination from filesystem to catalog (e.g., Unity Catalog) • enables multi-table transactions, enforced access controls, and server-side planning
Delta Kernel	Java and Rust libraries for building connectors	• Set of libraries for reading and writing Delta tables without understanding protocol details • replaces deprecated Delta Standalone; powers DuckDB, Flink, and custom connectors

Table 2: Read and Write Operations

The bread-and-butter DataFrame operations for getting data in and out of a Delta table. Beyond plain reads, appends, and overwrites, this is where time travel shows up — querying a table as of a version or timestamp — alongside streaming reads and writes with exactly-once guarantees and idempotent transaction IDs that keep duplicate writes from sneaking in.

Operation	Example	Description
Read Delta table	`spark.read.format("delta").load("/path")`	• Load latest table version as DataFrame • supports predicate pushdown and column pruning for efficient queries

The rest of this section is part of the full cheat sheet

Table 3: Table Manipulation

The SQL commands that change a table's data or shape — creating it, upserting with MERGE, running targeted UPDATE and DELETE, and evolving columns in place. Several of these lean on Delta features like deletion vectors and column mapping to avoid rewriting every Parquet file, and CONVERT TO DELTA is the on-ramp for migrating an existing plain-Parquet dataset.

Command	Example	Description
CREATE TABLE	`CREATE TABLE events (id LONG, ts TIMESTAMP)` `USING delta LOCATION '/data/events'`	• Define table schema in metastore • managed tables store data in default location, external use specified path

The rest of this section is part of the full cheat sheet

Table 4: Schema Management

Delta refuses mismatched writes by default — schema enforcement is what keeps a table from quietly corrupting. The techniques here are the controlled ways to change that schema anyway: opt into evolution to add or widen columns, decouple logical from physical names with column mapping, and reach for generated, identity, default, or variant columns when you want the table itself to compute or shape values.

Technique	Example	Description
Schema enforcement	Write with mismatched schema → fails	• Rejects writes with incompatible schemas by default • prevents accidental corruption from incorrect data types

The rest of this section is part of the full cheat sheet

Table 1: Core Concepts

Concept	Example	Description
Transaction log	`_delta_log/00000000000000000000.json`	• Append-only JSON log that records every table change • each commit creates a new log file numbered sequentially, enabling ACID guarantees and time travel
ACID transactions	Multiple writers commit simultaneously	• Atomicity, Consistency, Isolation, Durability via optimistic concurrency control • failed transactions roll back without affecting committed data
Parquet data files	`part-00000-<uuid>.snappy.parquet`	• Columnar storage format containing actual data • Delta adds metadata layer on top for transactions and versioning
Checkpoint	`_delta_log/00000000000000000010.checkpoint.parquet`	• Parquet snapshot of table state written every 10 commits (default) • accelerates metadata reads by avoiding replay of thousands of JSON log entries • v2 checkpoints available via `delta.checkpointPolicy = 'v2'`
Table protocol version	`minReaderVersion=3, minWriterVersion=7`	• Protocol defines minimum client capabilities required to read/write a table • higher versions unlock features; supports table features model for granular opt-in
Table features	`deletionVectors`, `columnMapping`, `v2Checkpoint`	• Opt-in capabilities that enhance table functionality • enabling a feature upgrades table protocol and may restrict older client access
Lakehouse architecture	Delta tables + compute engines	• Storage layer supporting both BI and ML workloads on a single copy of data • combines warehouse reliability with lake scale and cost
Catalog-managed tables	`delta.feature.catalogManaged = 'supported'`	• Shifts commit coordination from filesystem to catalog (e.g., Unity Catalog) • enables multi-table transactions, enforced access controls, and server-side planning
Delta Kernel	Java and Rust libraries for building connectors	• Set of libraries for reading and writing Delta tables without understanding protocol details • replaces deprecated Delta Standalone; powers DuckDB, Flink, and custom connectors

Table 2: Read and Write Operations

Operation	Example	Description
Read Delta table	`spark.read.format("delta").load("/path")`	• Load latest table version as DataFrame • supports predicate pushdown and column pruning for efficient queries

Table 3: Table Manipulation

Command	Example	Description
CREATE TABLE	`CREATE TABLE events (id LONG, ts TIMESTAMP)` `USING delta LOCATION '/data/events'`	• Define table schema in metastore • managed tables store data in default location, external use specified path

Table 4: Schema Management

Technique	Example	Description
Schema enforcement	Write with mismatched schema → fails	• Rejects writes with incompatible schemas by default • prevents accidental corruption from incorrect data types