Statistical inference tests are formal procedures for evaluating claims about population parameters using sample data. They form the backbone of evidence-based decision-making in fields from medicine to machine learning, enabling researchers to distinguish genuine effects from random noise. Each test comes with specific assumptions about data distribution, sample size, and measurement scale—choosing the right test requires understanding both your research question and your data's characteristics. A critical insight: no test "proves" a hypothesis true; instead, tests quantify the probability that observed results could occur by chance alone under a null hypothesis, guiding decisions about what conclusions the data support.
What This Cheat Sheet Covers
This topic spans 21 focused tables and 132 indexed concepts. Below is a complete table-by-table outline of this topic, spanning foundational concepts through advanced details.
Table 1: Core Hypothesis Testing Concepts
Before you can pick the right test, you need the shared vocabulary that every test is built on. These are the moving parts of any hypothesis test—the null and alternative you weigh against each other, the p-value and significance level that decide the verdict, and the two ways you can get it wrong (false positives and false negatives) along with the power and effect size that determine how often you'll actually catch a real effect.
| Concept | Example | Description |
|---|---|---|
H₀: μ = 50 | • Statement of no effect or no difference • the default assumption tested against | |
H₁: μ ≠ 50 | • Statement that contradicts H₀ • what researcher seeks evidence for | |
p = 0.032 | • Probability of obtaining observed data (or more extreme) if H₀ is true • lower values indicate stronger evidence against H₀ | |
α = 0.05 | • Threshold for rejecting H₀ • typical values are 0.05, 0.01, or 0.001 | |
t = 2.45, z = 1.96 | • Numerical summary of sample data under H₀ • compared to a reference distribution to obtain p-value | |
Reject H₀ when true | • False positive • probability equals α | |
Fail to reject H₀ when false | • False negative • probability denoted β |