Some changes are too big to inject with JavaScript. A rebuilt template, a different framework, a checkout on another platform, an entire redesign: there is nothing on the control page to overwrite, because the thing you want to test does not exist there. That is what split URL testing is for, and it arrives with a set of ways to ruin data that in-page testing simply does not have.
A split URL test, also called a redirect test, sends a share of visitors to a second address and compares outcomes between the two. Same statistics, same discipline, different plumbing. The plumbing is where these tests are won and lost, so the plumbing is all this post is about.
When the second URL is the right answer
Reach for a redirect test when the variant cannot be produced by editing the control’s DOM: a rebuilt page template, a new stack, a landing page living on a different system, or a redesign big enough that a JavaScript override would be slower than the page it replaces. Redesign work is the common case, and the staged rollout in our guide to redesigning without losing revenue is, underneath, a series of split URL tests with the traffic share turned up as evidence accumulates.
Do not reach for one when an in-page change would do the job. Every redirect costs latency, complicates identity and creates search housekeeping, and none of those costs are paid by an in-page variation. A headline test on a second URL is a self-inflicted injury with a project plan attached.
The SEO rules, in order
Search engines have been fine with split testing for well over a decade, provided you do not accidentally describe your test as something permanent. Four rules cover almost all of it:
- Use a 302, never a 301. A 301 is permanent and instructs crawlers to swap the indexed URL for the variant, which is precisely the outcome you do not want the week the test ends.
- Put a canonical tag on the variant pointing at the control URL. This is the single instruction telling a crawler which of the two addresses is the real page, and it is the one most often forgotten.
- Do not noindex the variant. A noindexed page that a canonical also points away from sends two contradictory signals where one clear signal would have done.
- Take the test down when it is finished. A redirect left running for a year stops being an experiment and becomes your site architecture, undocumented and inherited by whoever comes next.
Run the test for as long as the statistics need and not a week longer. A split URL test is a temporary state of your site, and the risk grows with how long you leave that state in place rather than with the fact of having run one at all.
Redirect latency is a bias, not an inconvenience
Here is the part that quietly ruins results. The control loads once. The variant loads, redirects, then loads again. When that redirect is performed client-side by a testing tool’s script, the extra cost commonly runs 300 to 600 ms, and we have measured 420 ms on a mid-range Android over 4G with the redirect firing in JavaScript after the control page had already started rendering. Every one of those milliseconds is charged to one arm only.
That is not noise, it is a handicap. Some share of visitors abandon during the extra hop, and they abandon before they can see the thing you were testing, so the variant ends up measured on a slightly more patient population than the control. It surfaces as a mild, plausible loss, which is far more dangerous than an obvious one: an obvious loss gets investigated, a mild one gets believed, and a redesign worth six figures gets cancelled on the strength of a redirect.
Two fixes. Do the redirect at the edge or on the server, where a 302 costs tens of milliseconds instead of hundreds and happens before anything renders. Failing that, measure it: record the redirect duration in your QA notes and treat any test carrying a three-figure gap as suspect until proven otherwise. On Shopify the practical options are narrower than elsewhere, and A/B testing on Shopify sets out which of them survive contact with the platform.
The QA that in-page tests never need
Add these to whatever pre-launch checklist you already run, because none of them can fail on a test that stays on one URL:
- Analytics identity survives the hop. A visitor who becomes a fresh session on the variant URL inflates your traffic and dismantles your funnel report at the same time.
- The variant is excluded from sitemaps, internal links and paid campaign destinations, so its only inbound traffic is the test itself.
- Query strings and UTM parameters are carried across the redirect, or attribution quietly reassigns the variant’s revenue to the wrong channel.
- Back-button behaviour is sane. A redirect that traps someone in a loop when they try to go back is a conversion killer that no report will ever name.
- Assignment is bucketed once and remembered. A returning visitor bounced into the opposite arm on their second session pollutes both sides of the test.
- The variant renders correctly for someone who arrives at it directly, from a bookmark or a shared link, rather than only for someone the redirect delivered there.
One check deserves its own moment, because redirect tests produce more of it than any other test type we run: sample ratio mismatch. Everyone lost in flight was counted as assigned and never counted as arrived, so the observed split drifts away from the one you configured. A test set to 50/50 that delivers 48.6 against 51.4 on 60,000 visitors is not close, it fails a chi-square check at p below 0.0001, and the result is void rather than noisy. Run that check on the first day of traffic, before you look at a single conversion, because the second day is when somebody screenshots the number.
What a good split URL test looks like
Server-side or edge redirect under 50 ms, a 302 status, a canonical on the variant pointing at the control, assignment verified inside the first hour, identity preserved across the hop, and a removal date written into the ticket. Six things, none of them clever, and the tests that go wrong are almost always missing three of them at once rather than being undone by anything subtle.
None of this makes redirect testing exotic. It makes it engineering, which is the honest description of most of what separates a test you can act on from a chart you can argue about. If you are about to test a rebuilt template and you would rather the plumbing were somebody else’s problem, that is exactly what our A/B test development work is, and you can start a conversation as soon as the design is ready.
