Free Speed & UX tool
A/B Test Significance Calculator
Find out if your variant really won, or how many visitors you need before you start.
- Free
- No sign-up
- Runs in your browser
Enter what each version got so far.
B wins with 98.7% confidence
B converts 16.0% better than A. A gap this size is unlikely to be chance.
- Relative uplift
- +16.0%
- Absolute change
- +0.80 pts
- p-value
- 0.0123
- 95% interval, B − A
- +0.17 to +1.43
Conversion rate, with 95% range
Bars show where each true rate most likely sits. When they barely overlap, the gap is real.
Two-sided two-proportion z-test. Decide your sample size before you start, and don’t stop the moment it turns green.
What it checks
Plain verdict
Whether B won, lost or needs more time, at 90, 95 or 99% confidence.
Conversion rates
Each version’s rate, with the range where the true rate most likely sits.
Uplift
The change from A to B, as a relative percentage and in percentage points.
p-value and interval
A two-sided z-test, with a confidence interval for the difference.
Sample size
Visitors per variant to detect the uplift you care about, at 80 or 90% power.
Test length
How many days that takes at your daily traffic.
Questions
What does statistically significant mean?
That a difference this big would rarely happen by chance if A and B were really the same. At 95% confidence, the p-value has to be under 0.05.
Can I stop the test as soon as it’s significant?
Better not. Checking every day and stopping at the first green result finds far more false winners. Work out the sample size first, then wait for it.
How is the sample size worked out?
With the standard formula for comparing two proportions: your current rate, the smallest uplift worth finding, the power and the significance level. Smaller uplifts need many more visitors.
Is my data sent anywhere?
No. Every number is calculated in your browser and nothing is stored.