Compare two groups' averages with a t-test

Two channels look different in average order value, and I can't tell if that's a real gap or just noise.

BeginnerRead 5 minDoing: an hour6 steps

What you'll have at the end

A written verdict, backed by a p-value and an effect size, on whether two channels' average order values genuinely differ

Before you start

You need

Two lists of order values in dollars, one per channel, pulled from the same time window with no order counted twice.

Not covered

Working out how many orders you'd need to collect before running the test, or comparing more than two groups at once.

Leans on

An A/B test isn't significant, and the team still needs a call

your own comparison comes back not significant and you still need to give your manager a next step

Your data fails normality; try a nonparametric test instead

one group's data is heavily skewed or full of outliers and the t-test's normality assumption looks shaky

Work out how big a sample an A/B test needs

you're planning the comparison before collecting any data and want to know how many orders to gather

Checked 26 Aug 2026