Data cleaning and validation is the workflow that turns a raw export from a spreadsheet, database, or third-party feed into numbers a team can actually act on. It sits between pulling data (SQL queries, API calls, downloads) and analyzing it — skip this step and duplicate rows, mismatched units, missing values, and biased samples quietly ride along into every chart and average built on top. The core mental model to keep: clean is not the same as accurate. A dataset can be perfectly formatted and still be systematically wrong if it came from an unreliable source or a skewed sample, so validation has to check the data's origin and representativeness, not just its structure. Because the same messy-data problems recur across Excel, SQL, and Python work, the examples below use whichever tool practitioners reach for first — and every cleaning decision is worth writing down, since the fastest way to lose a stakeholder's trust is being unable to explain how a number was produced.
What This Cheat Sheet Covers
This topic spans 10 focused tables and 95 indexed concepts. Below is a complete table-by-table outline of this topic, spanning foundational concepts through advanced details.
A jump-to index of every table row in this cheat sheet.
An interactive map of every table and concept in this topic.
Table 1: Evaluating Data Source Reliability
Before a single value gets touched, an analyst has to decide whether the source itself deserves trust — a beautifully clean dataset built on a biased or fabricated source is still worthless. These checks catch bad data at the door, before it costs any cleaning time at all.
| Criterion | Example | Description |
|---|---|---|
Census Bureau microdata (primary)vs. a news article citing it (secondary) | A primary source collected the data firsthand; a secondary source interprets or repackages it — trace a surprising number back to the primary source before citing it. | |
check who published the datasetand their history of accurate releases | An established statistical agency or peer-reviewed source with a track record of accuracy outranks an anonymous spreadsheet found online. | |
sample size, survey questions, andcollection dates are disclosed | • Reputable sources publish how the data was collected • missing methodology is a red flag, not a minor gap | |
a vendor-funded report sizingits own product category | Ask who paid for the data and what outcome benefits them — funding source shapes which numbers get published. |