We turn away testing engagements from stores under 30,000 monthly sessions. Not because small stores cannot improve, but because the mathematics of A/B testing punishes them, and selling a small store a testing programme is selling them a coin flip with a dashboard attached. That honesty costs us deals, so here is the playbook we hand those brands instead. It works, and most of it is free.
CRO for low-traffic websites is not a smaller version of enterprise CRO; it is a different discipline. Below roughly 30,000 sessions and 500 orders a month, most A/B tests cannot reach a trustworthy answer in a reasonable time, so the work shifts to heuristic fixes, qualitative research and honest before-and-after measurement. None of that requires a testing tool, and all of it moves revenue.
The maths against you
Run the numbers once and the argument ends. A store with 8,000 sessions a month converting at 2% wants to detect a 15% relative lift, which would take conversion to 2.3%. On standard assumptions that test needs roughly 37,000 visitors per arm, 74,000 in total, which at 8,000 sessions a month is over nine months of running a single test. Nine months of seasonality, promotions, stock changes and ad-budget shifts, all bleeding into a result that was supposed to isolate one change.
It gets worse, because a test starved of traffic does not just run slowly, it lies confidently: underpowered tests that do reach significance systematically exaggerate the effect they found, and a team peeking at a nine-month test weekly will stop it early on noise. Multiply that by a realistic programme, where most tests lose or go flat, and a small store can spend two years learning almost nothing. The full arithmetic is in our A/B testing sample size guide; the summary is that below the threshold, the tool that promises certainty mostly manufactures false confidence.
Fix the obvious first
The good news is that low-traffic stores rarely need a test to find money, because the defects are not subtle. A heuristic audit, a structured walk through the store on a real phone asking at every step what a first-time visitor cannot see, cannot find or cannot trust, surfaces most of them in a day: delivery costs hidden until checkout, a returns policy nowhere near the buy button, product photos that do not show scale, a four-second load on the money pages. None of these needs an experiment. A test tells you whether a change worked; when the change is repairing something visibly broken, you are allowed to just fix it.
Speed deserves its own line because it compounds every other problem: a slow product page suppresses every metric downstream of it, and the fixes, compressing images, removing dead apps, deferring scripts, are diagnosable from a free report and need no experiment to justify.
Qualitative research scales down
Testing needs traffic; talking to humans does not. Five moderated user tests, real people from your audience attempting a purchase while thinking aloud, will surface the biggest usability problems on the store for a few hundred pounds, and the findings repeat so reliably that the sixth test is mostly confirmation. Add session recordings and heatmaps, which work at any traffic level and are covered step by step in our heatmaps and session recordings guide, a one-question exit poll, and an hour reading support tickets and chat logs, and you have a research stack a store with 3,000 sessions can run every quarter. The point is not statistical rigour, it is watching five people fail to find your size guide and never needing a p-value to act.
Before and after, without fooling yourself
When you do change something significant, measure it as an honest quasi-experiment. Ship one change at a time, compare four full weeks after against four full weeks before, and hold the comparison honest: same traffic mix, no overlapping promotions, no seasonal cliff between the windows, every change and campaign annotated in your analytics. Read the step the change targeted rather than blended conversion; a checkout fix should move checkout completion, and blended conversion is noisy enough to hide it. Guardrails matter as much as the metric: watch refund rate, average order value and support volume over the same windows, so a change that lifts conversion by quietly attracting the wrong orders gets caught.
One store in our archive moved delivery costs into the cart and watched conversion go from 2.0% in the four weeks before to 2.2% in the four weeks after, a 10% relative change. Is that proof? No, and that is the honest limit of the method: before-and-after can be fooled by anything else that changed. It is evidence, it is directionally useful, and stacked with the qualitative findings it is usually enough to act on. What it is not is a licence to claim precision, so keep the language honest too: improved, not proved.
Copy the boring winners
The other substitute for testing is other people’s tests. Across our archive of 1640+ ecommerce experiments, a handful of patterns win so consistently that they function as near-universal priors, and a low-traffic store should simply adopt them:
- Show total costs, delivery included, in the cart or earlier; surprise fees at checkout are the most reliable conversion killer we measure.
- Default to guest checkout, and offer account creation after the order rather than before it.
- Add express payment wallets; on mobile they routinely carry a third or more of orders.
- Put the returns window and the delivery date next to the buy button, not in the footer.
- Cut every checkout form field that cannot justify itself.
- State delivery as a date, arrives Thursday, rather than a speed.
Adopting a proven pattern without a test carries some risk that it is wrong for your store. Carrying on with a known-bad pattern carries more.
The graduation thresholds
Our public rule of thumb is 30,000 monthly sessions or 500 monthly orders: around that level, a test on your highest-traffic template can read a moderate effect in four to six weeks, which is where testing starts to pay rent. Graduating does not mean testing everything; it means testing the few changes that are genuinely uncertain and expensive to get wrong, on the pages with the most traffic, while everything else stays on the playbook above.
Until then, the leverage is elsewhere, and the honest ranking for a small store is traffic and offer first, obvious fixes second, research third, testing last. If you want a second pair of eyes on where your store leaks, get in touch: an audit works at any size, even when a test programme does not.
