Data Analysis Cheat Sheet

Data analysis is the systematic process of inspecting, cleaning, transforming, and modeling data to discover useful information and support decision-making. It encompasses exploratory data analysis (EDA), data cleaning, wrangling, and feature engineering—the critical preparatory steps that transform raw data into analysis-ready datasets. While often associated with statistics and machine learning, data analysis fundamentally serves as the bridge between raw observations and actionable insights. A key insight: spending adequate time on quality data preparation typically has a larger impact on model performance than algorithm selection itself—well-prepared data enables simpler models to outperform complex ones trained on poor data. Modern workflows additionally use SHAP-based feature importance, automated drift detection, and schema validation tools to ensure pipelines remain robust and production-ready.

What This Cheat Sheet Covers

This topic spans 15 focused tables and 152 indexed concepts, 126 flashcards. Below is a complete table-by-table outline of this topic, spanning foundational concepts through advanced details.

Table 1: Exploratory Data Analysis (EDA) TechniquesTable 2: Data Cleaning TechniquesTable 3: Outlier Detection and TreatmentTable 4: Data Transformation and ScalingTable 5: Feature Encoding for Categorical VariablesTable 6: Feature Engineering TechniquesTable 7: Dimensionality Reduction and Feature SelectionTable 8: Data Aggregation and GroupingTable 9: Data Reshaping and MergingTable 10: Statistical Analysis and Hypothesis TestingTable 11: Data Wrangling OperationsTable 12: Data Quality and ValidationTable 13: Data Visualization for AnalysisTable 14: Sampling and Balancing TechniquesTable 15: Automated EDA and Monitoring Tools

Quick IndexSubscribe to unlock

A jump-to index of every table row in this cheat sheet.

Mind MapSubscribe to unlock

An interactive map of every table and concept in this topic.

Table 1: Exploratory Data Analysis (EDA) Techniques

Before you clean or model anything, you look — and these are the moves that turn a raw DataFrame into an understood one. From quick describe() summaries to Q-Q plots, correlation heatmaps, and one-line automated profiling reports, each technique answers a different question about shape, spread, missingness, and how variables relate. Working from single variables up to multivariate views is the natural order to read them in.

Technique	Example	Description
summary statistics	`df.describe()` `df.info()`	• Computes descriptive measures—count, mean, std, min, quartiles, max for numerical columns • `info()` shows data types and null counts.
value counts	`df['category'].value_counts(normalize=True)`	• Counts frequency of unique values in categorical columns • `normalize=True` returns proportions instead of counts.
univariate analysis	`df['age'].hist(bins=30)` `df['category'].value_counts()`	• Examines single variable distributions using histograms, box plots, and frequency counts • reveals central tendency, spread, and outliers for one feature at a time.
bivariate analysis	`df.plot.scatter(x='age', y='income')` `df.groupby('region')['sales'].mean()`	• Explores relationships between two variables through scatter plots, grouped aggregations, and cross-tabulations • identifies correlations and patterns.
multivariate analysis	`sns.pairplot(df, hue='target')` `df.corr()`	• Examines interactions among three or more variables using pair plots, correlation matrices, and heatmaps • reveals complex dependencies.
correlation analysis	`df.corr()` `sns.heatmap(df.corr(), annot=True)`	• Calculates pairwise correlation coefficients between numerical features • heatmap visualization reveals multicollinearity and feature relationships.
distribution analysis	`from scipy.stats import skew, kurtosis` `skew(df['col']), kurtosis(df['col'])`	• Assesses shape characteristics of data distributions • skewness measures asymmetry (positive = right-tailed), kurtosis measures tail heaviness.
KDE plot	`sns.kdeplot(data=df, x='col', hue='group', fill=True)`	• Plots continuous probability density as a smooth curve • less cluttered than histograms for comparing multiple distributions • bivariate version reveals joint density.
missing value heatmap	`import missingno as msno` `msno.heatmap(df)` `msno.matrix(df)`	• Visualises patterns of missing data across columns and rows • matrix shows nullity co-occurrence; heatmap shows correlation of missingness between columns.
Q-Q plot	`from scipy.stats import probplot` `probplot(df['col'], plot=plt); plt.show()`	• Compares sample quantiles against a theoretical distribution (default: normal) • points on the reference line indicate a good fit; deviations reveal skewness or heavy tails.
data profiling	`from ydata_profiling import ProfileReport` `ProfileReport(df).to_file("report.html")`	• Generates comprehensive automated reports with statistics, distributions, correlations, missing values, and data quality warnings • replaces manual EDA.

Table 2: Data Cleaning Techniques

Real datasets arrive messy — nulls, duplicates, wrong types, and inconsistent strings — and this is the toolkit for fixing each. Notice the choices around missing values: deleting rows is fast but throws away signal, while KNN and iterative (MICE) imputation reconstruct gaps from the patterns in other columns. The rest standardizes types and text so downstream operations behave predictably.

Technique	Example	Description
missing value detection	`df.isnull().sum()` `df.isna().sum()`	• Identifies null or NaN entries in each column • essential first step before choosing imputation strategy.

The rest of this section is part of the full cheat sheet

Table 3: Outlier Detection and Treatment

Extreme values can be genuine signal or pure noise, and your method depends on which. Statistical rules like IQR and Z-score work column by column, while isolation forest, LOF, and DBSCAN catch anomalies across many dimensions at once. The treatment side — winsorizing, capping, or outright removal — is a separate decision about whether to tame those points or drop them.

Method	Example	Description
IQR method	`Q1, Q3 = df['col'].quantile([0.25, 0.75])` `IQR = Q3 - Q1` `lower, upper = Q1 - 1.5IQR, Q3 + 1.5IQR`	• Defines outliers beyond 1.5 × IQR from quartiles • robust to extreme values, works well for skewed data • most common threshold.

The rest of this section is part of the full cheat sheet

Table 4: Data Transformation and Scaling

Many algorithms misbehave when features sit on wildly different scales or have skewed distributions, so you reshape them first. Scaling methods — standardization, min-max, robust — put features on comparable footing, while log, Box-Cox, and Yeo-Johnson transforms pull long tails back toward symmetry. Knowing which handles outliers, zeros, or negatives saves you from a transform that quietly breaks.

Technique	Example	Description
standardization (Z-score)	`from sklearn.preprocessing import StandardScaler` `StandardScaler().fit_transform(df)`	• Centers data to mean=0, std=1 • preserves distribution shape • required for algorithms sensitive to feature magnitude (SVM, neural networks).

The rest of this section is part of the full cheat sheet

Data Analysis Cheat Sheet

Table 1: Exploratory Data Analysis (EDA) Techniques

Technique	Example	Description
summary statistics	`df.describe()` `df.info()`	• Computes descriptive measures—count, mean, std, min, quartiles, max for numerical columns • `info()` shows data types and null counts.
value counts	`df['category'].value_counts(normalize=True)`	• Counts frequency of unique values in categorical columns • `normalize=True` returns proportions instead of counts.
univariate analysis	`df['age'].hist(bins=30)` `df['category'].value_counts()`	• Examines single variable distributions using histograms, box plots, and frequency counts • reveals central tendency, spread, and outliers for one feature at a time.
bivariate analysis	`df.plot.scatter(x='age', y='income')` `df.groupby('region')['sales'].mean()`	• Explores relationships between two variables through scatter plots, grouped aggregations, and cross-tabulations • identifies correlations and patterns.
multivariate analysis	`sns.pairplot(df, hue='target')` `df.corr()`	• Examines interactions among three or more variables using pair plots, correlation matrices, and heatmaps • reveals complex dependencies.
correlation analysis	`df.corr()` `sns.heatmap(df.corr(), annot=True)`	• Calculates pairwise correlation coefficients between numerical features • heatmap visualization reveals multicollinearity and feature relationships.
distribution analysis	`from scipy.stats import skew, kurtosis` `skew(df['col']), kurtosis(df['col'])`	• Assesses shape characteristics of data distributions • skewness measures asymmetry (positive = right-tailed), kurtosis measures tail heaviness.
KDE plot	`sns.kdeplot(data=df, x='col', hue='group', fill=True)`	• Plots continuous probability density as a smooth curve • less cluttered than histograms for comparing multiple distributions • bivariate version reveals joint density.
missing value heatmap	`import missingno as msno` `msno.heatmap(df)` `msno.matrix(df)`	• Visualises patterns of missing data across columns and rows • matrix shows nullity co-occurrence; heatmap shows correlation of missingness between columns.
Q-Q plot	`from scipy.stats import probplot` `probplot(df['col'], plot=plt); plt.show()`	• Compares sample quantiles against a theoretical distribution (default: normal) • points on the reference line indicate a good fit; deviations reveal skewness or heavy tails.
data profiling	`from ydata_profiling import ProfileReport` `ProfileReport(df).to_file("report.html")`	• Generates comprehensive automated reports with statistics, distributions, correlations, missing values, and data quality warnings • replaces manual EDA.

Table 2: Data Cleaning Techniques

Technique	Example	Description
missing value detection	`df.isnull().sum()` `df.isna().sum()`	• Identifies null or NaN entries in each column • essential first step before choosing imputation strategy.

Table 3: Outlier Detection and Treatment

Method	Example	Description
IQR method	`Q1, Q3 = df['col'].quantile([0.25, 0.75])` `IQR = Q3 - Q1` `lower, upper = Q1 - 1.5IQR, Q3 + 1.5IQR`	• Defines outliers beyond 1.5 × IQR from quartiles • robust to extreme values, works well for skewed data • most common threshold.

Table 4: Data Transformation and Scaling

Technique	Example	Description
standardization (Z-score)	`from sklearn.preprocessing import StandardScaler` `StandardScaler().fit_transform(df)`	• Centers data to mean=0, std=1 • preserves distribution shape • required for algorithms sensitive to feature magnitude (SVM, neural networks).