Data cleaning and validation is the workflow that turns a raw export from a spreadsheet, database, or third-party feed into numbers a team can actually act on. It sits between pulling data (SQL queries, API calls, downloads) and analyzing it — skip this step and duplicate rows, mismatched units, missing values, and biased samples quietly ride along into every chart and average built on top. The core mental model to keep: clean is not the same as accurate. A dataset can be perfectly formatted and still be systematically wrong if it came from an unreliable source or a skewed sample, so validation has to check the data's origin and representativeness, not just its structure. Because the same messy-data problems recur across Excel, SQL, and Python work, the examples below use whichever tool practitioners reach for first — and every cleaning decision is worth writing down, since the fastest way to lose a stakeholder's trust is being unable to explain how a number was produced.
What This Cheat Sheet Covers
This topic spans 10 focused tables and 95 indexed concepts, 95 flashcards. Below is a complete table-by-table outline of this topic, spanning foundational concepts through advanced details.
A jump-to index of every table row in this cheat sheet.
An interactive map of every table and concept in this topic.
Table 1: Evaluating Data Source Reliability
Before a single value gets touched, an analyst has to decide whether the source itself deserves trust — a beautifully clean dataset built on a biased or fabricated source is still worthless. These checks catch bad data at the door, before it costs any cleaning time at all.
| Criterion | Example | Description | |
|---|---|---|---|
Census Bureau microdata (primary)vs. a news article citing it (secondary) | • A primary source collected the data firsthand • a secondary source interprets or repackages it — trace a surprising number back to the primary source before citing it | ||
check who published the datasetand their history of accurate releases | An established statistical agency or peer-reviewed source with a track record of accuracy outranks an anonymous spreadsheet found online. | ||
sample size, survey questions, andcollection dates are disclosed | • Reputable sources publish how the data was collected • missing methodology is a red flag, not a minor gap | ||
a vendor-funded report sizingits own product category | Ask who paid for the data and what outcome benefits them — funding source shapes which numbers get published. | ||
a 2019 industry survey usedto forecast 2026 | • Stale data can be worse than no data in fast-moving domains • always check the last-updated date before trusting a figure | ||
confirm a statistic against twoindependent sources | • Agreement across independent sources raises confidence • a single uncorroborated number should be treated cautiously | ||
National Statistics Office censusor labor force survey | National statistical offices and regulators are usually the most rigorously audited source available for a given domain. | ||
volunteer-submitted air qualityreadings | • Can fill genuine gaps in official coverage • carries higher risk of inconsistent collection methods and self-selection |