ML Data Management and Data-Centric AI Cheat Sheet

Updated 2026-05-18

Next Topic: ML for Tabular Data Cheat Sheet

🧠Study flashcards on this topic89 cards · spaced repetition→

Data-centric AI shifts focus from model architecture optimization to systematic improvement of training data quality, diversity, and consistency. Modern ML systems require robust data management practices spanning versioning, validation, augmentation, documentation, and governance to ensure reproducible, ethical, and high-performing models at scale.

What This Cheat Sheet Covers

This topic spans 13 focused tables and 91 indexed concepts, 89 flashcards. Below is a complete table-by-table outline of this topic, spanning foundational concepts through advanced details.

Table 1: Data-Centric AI Core PrinciplesTable 2: Data Versioning and Lineage Tracking ToolsTable 3: Data Quality Frameworks and MetricsTable 4: Data Validation SystemsTable 5: Dataset Documentation StandardsTable 6: Annotation Quality and Label Noise HandlingTable 7: Data Augmentation Techniques by ModalityTable 8: Synthetic Data Generation MethodsTable 9: Dataset Balancing and Sampling StrategiesTable 10: Feature Engineering and PreprocessingTable 11: Data Pipeline Orchestration and TestingTable 12: Data Drift Detection and MonitoringTable 13: Data Governance, Privacy, and Collaboration

Quick IndexSubscribe to unlock

A jump-to index of every table row in this cheat sheet.

Mind MapSubscribe to unlock

An interactive map of every table and concept in this topic.

Table 1: Data-Centric AI Core Principles

Data-centric AI prioritizes improving dataset quality over model complexity, recognizing that better data yields better models. This approach emphasizes systematic data engineering practices including consistency checking, error detection, and iterative dataset refinement as the primary driver of model performance improvements.

Principle	Example	Description
Data Quality Over Model Complexity	Focus on fixing mislabeled samples in training data rather than adding more layers to the model	Andrew Ng's data-centric AI approach: small, high-quality datasets often outperform large, noisy datasets when paired with modern architectures
Label Consistency	Standardize annotation guidelines so all annotators label "ambiguous" examples the same way	• Reduces label noise that degrades model generalization • critical for multi-annotator datasets
Systematic Data Augmentation	Generate synthetic edge cases to balance underrepresented classes in medical imaging datasets	• Augmentation as a data engineering strategy rather than a model training trick • focuses on principled dataset expansion
Data Debugging	Use Cleanlab to detect outliers, label errors, and near-duplicates in training data	• Automated data quality checks replace manual data inspection • identifies systematic annotation errors
Iterative Data Improvement	Deploy model → analyze failure modes → collect targeted data for failures → retrain	Closed-loop dataset refinement driven by production error analysis rather than static dataset collection
Data Slicing	Evaluate model separately on demographic subgroups to detect performance disparities	Uncovers hidden biases where aggregate accuracy masks poor performance on critical subpopulations

Table 2: Data Versioning and Lineage Tracking Tools

Data versioning systems enable reproducible ML experiments by tracking datasets, preprocessing scripts, and model artifacts across pipeline iterations. Lineage tracking provides audit trails showing how raw data transforms into features and predictions, essential for debugging, compliance, and collaboration.

Tool	Example	Description
DVC (Data Version Control)	`dvc add data/train.csv` `dvc push`	• Git-like versioning for datasets and models • stores large files in remote storage (S3, GCS, Azure) while tracking pointers in Git

The rest of this section is part of the full cheat sheet

Table 3: Data Quality Frameworks and Metrics

Data quality frameworks systematically measure and enforce standards for completeness, accuracy, consistency, timeliness, and validity across ML datasets. Automated quality checks integrated into data pipelines prevent bad data from degrading model performance in production.

Framework	Example	Description
Great Expectations	`expect_column_values_to_be_between("age", min_value=0, max_value=120)`	• Test-driven data validation • expectation suites define required statistical properties • generates data quality reports

The rest of this section is part of the full cheat sheet

Table 4: Data Validation Systems

Data validation systems enforce schema contracts, detect anomalies, and ensure incoming data matches expected distributions before training or serving. Validation layers act as gatekeepers preventing corrupted or drifted data from propagating through ML pipelines.

System	Example	Description
JSON Schema	`{"type": "object", "properties": {"age": {"type": "integer", "minimum": 0}}, "required": ["age"]}`	• Declarative validation for JSON data • defines required fields, types, ranges, and patterns

The rest of this section is part of the full cheat sheet

Table 1: Data-Centric AI Core Principles

Principle	Example	Description
Data Quality Over Model Complexity	Focus on fixing mislabeled samples in training data rather than adding more layers to the model	Andrew Ng's data-centric AI approach: small, high-quality datasets often outperform large, noisy datasets when paired with modern architectures
Label Consistency	Standardize annotation guidelines so all annotators label "ambiguous" examples the same way	• Reduces label noise that degrades model generalization • critical for multi-annotator datasets
Systematic Data Augmentation	Generate synthetic edge cases to balance underrepresented classes in medical imaging datasets	• Augmentation as a data engineering strategy rather than a model training trick • focuses on principled dataset expansion
Data Debugging	Use Cleanlab to detect outliers, label errors, and near-duplicates in training data	• Automated data quality checks replace manual data inspection • identifies systematic annotation errors
Iterative Data Improvement	Deploy model → analyze failure modes → collect targeted data for failures → retrain	Closed-loop dataset refinement driven by production error analysis rather than static dataset collection
Data Slicing	Evaluate model separately on demographic subgroups to detect performance disparities	Uncovers hidden biases where aggregate accuracy masks poor performance on critical subpopulations

Table 2: Data Versioning and Lineage Tracking Tools

Tool	Example	Description
DVC (Data Version Control)	`dvc add data/train.csv` `dvc push`	• Git-like versioning for datasets and models • stores large files in remote storage (S3, GCS, Azure) while tracking pointers in Git

Table 3: Data Quality Frameworks and Metrics

Framework	Example	Description
Great Expectations	`expect_column_values_to_be_between("age", min_value=0, max_value=120)`	• Test-driven data validation • expectation suites define required statistical properties • generates data quality reports

Table 4: Data Validation Systems

System	Example	Description
JSON Schema	`{"type": "object", "properties": {"age": {"type": "integer", "minimum": 0}}, "required": ["age"]}`	• Declarative validation for JSON data • defines required fields, types, ranges, and patterns