A/B Test Sample Size & Duration Calculator
Most calculators tell you how many visitors you need. That is the easy half. This one tells you how many weeks that will take at your traffic, and whether the test is worth starting at all.
Your test
The rate the page converts at today. Use the figure for the page under test, not the whole site.
The minimum detectable effect. Relative means a share of your baseline, so 10% on a 2.5% rate is a move to 2.75%.
Sessions reaching the page under test, not sitewide. Leave it blank if you do not know it yet and you will still get the sample size.
Every extra variant splits the same traffic further and adds a comparison to correct for.
What that means
Borderline: about 6 weeks.
At 64,199 visitors per variant this runs long enough that the season, your prices and your traffic mix will all change underneath it. Test a bolder change, or a page with more traffic.
- Visitors per variant
- 64,199
- Total visitors
- 128,398
- Test duration
- 6 weeks
- Conversions per variant
- 1,605
What a bolder change would buy you
| Relative lift | Per variant | Duration |
|---|---|---|
| 5% | 250,846 | 21 weeks |
| 10%Your setting | 64,199 | 6 weeks |
| 15% | 29,193 | 3 weeks |
| 20% | 16,792 | 2 weeks |
The same test at four different effect sizes. This is usually the most useful thing on the page: doubling the size of the change you are testing cuts the time by far more than double.
To detect a 10% relative lift from a 2.5% baseline at 95% significance and 80% power, you need 64,199 visitors per variant, 128,398 in total. That is about 6 weeks at your traffic.
How this calculator worksA fixed-horizon two-proportion z-test with a pooled variance term, the same method Evan Miller made the industry standard. Every assumption is listed below.
The sample size per variant comes from the standard two-proportion comparison. Writing p1 for your baseline and p2 for the rate you want to be able to detect, and z for the inverse standard normal:
- p2 is p1 x (1 + MDE) in relative mode, or p1 + MDE in absolute mode.
- The pooled term uses the average of the two rates, the unpooled term uses each rate separately, and the whole thing is divided by the squared difference between them.
- A two-sided hypothesis halves alpha before it reaches z. A one-sided hypothesis does not, which is why it always looks cheaper.
- Bonferroni, when it is on, divides alpha again by the number of comparisons against control, which is one fewer than your variant count.
- Duration is total sample divided by the traffic actually entering the test, rounded up to whole weeks.
What this assumes, and where it stops being true
This is a fixed-horizon calculation. It assumes you decide the sample size first, run to it, and look once. If you check the result daily and stop when it goes green, the false-positive rate is far higher than the significance level says, and no sample size fixes that. It also assumes each visitor is independent and counted once, that the variants differ only in what you changed, and that the baseline you typed is stable. Seasonality, a price change or a traffic-source shift mid-test all break the last assumption, which is the real reason long tests are worse than short ones rather than just slower.
It also does not report confidence in a result you have already run. That is a different calculation and a different page, and treating one minus the p-value as a confidence level is the single most common way A/B testing goes wrong. We have written about what significance actually tells you separately.
The maths, implemented from scratch
There is no statistics library behind this. The inverse normal uses Acklam’s rational approximation, accurate to better than one part in a billion, and the normal CDF uses the Abramowitz and Stegun series. Both are unit-tested against published values, and the sample-size function is checked against ten reference cases so the output matches the calculators practitioners already trust. If a number here disagrees with one you have from elsewhere, the usual cause is a one-sided setting on one of them.
How to use the results
The headline number is the duration, not the sample size, and that is deliberate. Nobody has ever cancelled a test because it needed 64,000 visitors per variant. Plenty of teams have quietly abandoned one four months in. Weeks are the unit the decision is actually made in, so weeks are what the tool leads with.
If the verdict is green
Four weeks or less means the maths is not your problem, so stop optimising it. The thing most likely to waste the next month is a weak hypothesis, not a slightly suboptimal power setting. Write down before you start what result would make you ship, what result would make you roll back, and what you would conclude from a flat result, because a flat result is the most likely outcome and the one teams handle worst. Then leave the test alone until it reaches the sample size. Checking daily and stopping early is the fastest way to turn a valid test into a coin flip with extra steps.
If the verdict is amber
Five to eight weeks is the range where tests technically finish and still fail to tell you anything. Over two months your traffic mix shifts, you run a promotion, a competitor changes their pricing, and the two groups you are comparing are no longer drawn from the same population. The honest fix is almost never to run it longer. It is to test a bigger change.
The sensitivity table is there for exactly this moment. Look at what happens to the duration when the effect size goes from 10% to 20%: the time does not halve, it drops by roughly three quarters, because sample size scales with the square of the effect. A test of a genuinely different page beats a test of a button colour by a margin most teams underestimate. If you cannot make the change bolder, move the test to a template with more traffic, or work out the sample size for a different page before committing to this one.
If the verdict is red
This is the answer most calculators will not give you. Above eight weeks, the test does not fail because the statistics are wrong. It fails because nothing about your store holds still that long, so whatever the result says at the end, it is not answering the question you asked at the start. Running it anyway costs you the traffic and gives you a number you should not act on.
Under roughly 30,000 sessions a month to a single template, split testing is usually the wrong tool rather than a tool you are using badly. That does not mean you cannot improve conversion. It means the evidence has to come from somewhere other than a significance test: session recordings, checkout drop-off, unmoderated testing on five people, and fixing the things that are plainly broken without needing proof they are broken. We have written up what to do instead at low traffic in full.
The numbers underneath the verdict
Conversions per variant is worth a look even when the duration is comfortable. Below about 150 conversions per arm, a significant result rests on a handful of orders, and a handful of orders is well inside the range that a single wholesale customer or one bad delivery week can move. The sample size will say you are done. The result will still be fragile.
Traffic allocation changes the calendar and nothing else. Ramping to 50% does not make the test cheaper, it makes it twice as long, which is a real cost rather than a safety measure. And if you are running more than two variants, leave the correction on. Three variants means two comparisons against control, and two chances to find a false winner is meaningfully worse than one.
Common questions
It depends entirely on your baseline conversion rate and the size of the effect you want to detect, which is why a single answer like "1,000 per variant" is always wrong. A store converting at 2.5% that wants to detect a 10% relative lift needs about 64,000 visitors per variant at 95% significance and 80% power. The same store looking for a 20% lift needs about 16,800, because sample size scales with the square of the effect size.
Until it reaches the sample size you calculated, and no longer. In practice, aim for a whole number of weeks so every day of the week is represented equally, and do not stop early because the result looks good. Under four weeks is comfortable, five to eight is where seasonality starts to distort the comparison, and beyond eight weeks the test is usually answering a different question from the one you asked.
It is the smallest improvement the test is powered to find. It is not a prediction of what you will get. Set it to the smallest lift that would actually change what you do: if a 3% improvement would not justify shipping the change, do not power the test to detect one. Most teams set it far too low, which is the single most common reason a test needs traffic the store does not have.
Two-sided, in almost every case, which is why it is the default here. One-sided tests only ask whether the variant is better, not whether it is worse, and they need roughly 20% less traffic as a result. That discount is real but you are paying for it by giving up the ability to detect a loss, and shipping a change that quietly costs you money is a worse outcome than a slightly longer test.
Usually because the two are set to different hypotheses. A one-sided setting returns roughly 20% less traffic than a two-sided one for the same inputs, and several popular tools default to one-sided without saying so. The other common cause is that some platforms plan with a fixed-horizon calculation and then analyse with a sequential or Bayesian engine, so their own two numbers do not agree with each other either.
Test bigger changes, test on higher-traffic templates, or stop testing and use other evidence. Below roughly 30,000 sessions a month to a single template, most A/B tests will not reach significance in a useful timeframe. That is not a reason to stop improving conversion, it is a reason to get the evidence from session recordings, checkout analytics and small-sample usability testing instead, and to fix obviously broken things without demanding statistical proof first.
Put this calculator on your own site
Free to embed, no attribution deal to sign and no tracking script. Paste the snippet wherever you write about test planning. It works at any width and keeps a link back here.
<iframe src="https://optyv.com/tools/ab-test-calculator/embed" width="100%" height="880" style="border:0;max-width:760px" title="A/B test sample size and duration calculator" loading="lazy"></iframe>
<p>Calculator by <a href="https://optyv.com/tools/ab-test-calculator">Optyv</a></p>The iframe is a fixed 880px tall in this version. If your column is narrow, the calculator scrolls inside the frame rather than clipping.
Citing a result?
If you are quoting a number from this calculator in a post, this badge says where it came from.
<a href="https://optyv.com/tools/ab-test-calculator"><img src="https://optyv.com/badge/ab-testability-checked.svg" alt="A/B testability checked with Optyv" width="220" height="40"></a>If the answer was a number you did not want
We build and run tests for stores where the maths is tight, and we say so when testing is the wrong call. Either way you get the honest version first.
Talk to us ↗