New: Cookbooks and AI ExplanationsStep-by-Step recipes to solve problems connected to Roadmaps and Cheat Sheets. Need more details? Use AI buttons for structured and simple explanations with concrete examples throughout the whole platform.Take a look
My test result came back not significant, and my manager still wants to know what to do next.
What you'll have at the end
A short writeup, ready to send to the team, stating what the interval does and doesn't rule out, why the test came back inconclusive, and one concrete next step.
You need
A completed A/B test whose primary metric came back not statistically significant, with each arm's sample size and conversion count, or an event log you can split by segment.
Not covered
Redesigning the test itself, its traffic split, guardrail metrics, or stopping rule, and working out exactly why one segment underperformed once you've flagged it.
Compare two groups' averages with a t-test
Go there if your metric is a continuous average, like order value or session length, instead of a conversion rate; the same read-the-interval logic runs on a different formula.
Work out how big a sample an A/B test needs
Go there before your next test if the sample-size gap this recipe uncovers keeps happening, so you plan for it upfront instead of diagnosing it after the fact.
State a confidence interval without overclaiming it
Go there for the fuller ground rules on what a confidence interval does and doesn't promise, beyond the one interval this recipe walks through.
Checked 26 Aug 2026
Part of the Statistics Fundamentals cookbook