Most CRO programmes do not fail on statistics. They die of neglect somewhere around week eight, when the first three tests lose, the champion gets pulled onto another project, and everyone quietly goes back to shipping on opinion. We have watched it happen from the inside more times than we can count across nine years of building experiments, and the pattern is so repeatable that the fix can be written down as an operating system. This is that operating system: what a CRO programme actually is, who runs it, on what rhythm, and how it survives its first losing streak.
A CRO programme is four things and nothing more: one named owner, one weekly cadence, one prioritised backlog and one archive of every result. Everything else, the tooling, the statistics, the test design, hangs off that frame, and none of it saves a programme that lacks the frame. This guide deliberately stays at the organisational layer; the statistics and the method are covered in their own guides, linked where they belong, because re-teaching them here would only bury the part that actually kills programmes.
Why programmes die inside 90 days
Run the autopsy on any abandoned programme and the same three organs failed. There was no owner: experimentation was everyone’s job, which made it nobody’s, and the moment a product launch or a sale demanded attention, testing was the thing that slipped. There was no cadence: tests launched when someone remembered, results were read when someone asked, and the gaps stretched quietly from days to months. And expectations were set on winners rather than learnings: somebody sold the programme internally on a highlight reel of case studies, so when the first quarter came in at the realistic rate, roughly one clear winner in four tests, the budget conversation went badly.
Notice what is not on that list: bad statistics, wrong tools, insufficient traffic. Individual tests fail for technical reasons, and we keep a separate guide to those failure modes. Programmes fail organisationally. A team can run every individual test cleanly and still lose the programme, because nobody owned the boring machinery of keeping it moving.
The minimum viable programme
The owner is a single named person with the authority to ship changes and the time to spend, two to four protected hours a week at minimum. The cadence is one recurring weekly slot that never moves. The backlog is a scored list of test ideas, each written as a hypothesis with evidence attached. The archive is a record of every test ever run, winners, losers and flat results alike, with what was learned. A solo founder can stand this up in a fortnight with a spreadsheet; an enterprise can fail to stand it up with a six-figure platform. The frame is the programme.
What the owner actually does week to week is unglamorous project management: chase blocked builds, keep the backlog scored, make sure a finished test gets read rather than left running, and defend the weekly slot from everything that wants it. The job title does not matter, founder, head of growth, ecommerce manager; the two non-negotiables are metric authority and protected hours, because a programme owned by someone with neither is a hobby.
Who owns it at each size
At solo-founder scale the owner is you, the cadence is one protected morning, and the honest constraint is traffic; below the testing threshold, run the same frame around before-and-after measurement instead of split tests. At marketing-team scale the owner should be whoever answers for the conversion metric, not whoever had spare hours, because an owner without metric authority cannot defend the programme when priorities collide. At the point where testing velocity matters, the roles split: one person owns the programme, others build the experiments.
That build layer is the most outsourceable piece of the whole system, because it is specialist engineering with clear acceptance criteria, which is precisely what a dedicated A/B test development service sells. The decisions, what to test and what the results mean, should stay in-house even when the building does not; our guide to outsourcing the layers maps the engagement models and the red flags on both sides.
The cadence that keeps it alive
The rhythm we install everywhere is research Mondays, readout Fridays. Monday, thirty minutes: review what is live, check the guardrails, and feed the backlog with whatever last week’s data surfaced. Friday, thirty minutes: read out any test that has reached its planned sample, decide ship, iterate or kill, and log the result in the archive before anyone leaves the call. Two half-hour meetings sounds too light to matter, which is exactly the point; a cadence survives because it is cheap. The dashboard of a healthy programme is correspondingly boring: a backlog of 34 scored ideas, three experiments live, research Monday and readout Friday sitting in the calendar like standing furniture.
A readout has a fixed shape: what we believed, what we ran, what happened, what we decided. The decision vocabulary is deliberately small, ship, iterate or kill, and the metric that decides is the one the hypothesis named in advance. Ban the phrase “let it run a bit longer”; a test that reached its planned sample is finished, and extending it because the answer disappoints is how programmes teach themselves to peek. Quarterly, one longer session zooms out: which themes are winning, which templates are exhausted, what next quarter’s focus is, and whether velocity justifies cost. Booking that review in week one removes the ambient anxiety of an unscheduled reckoning.
The cadence is also what survives the losing streak that kills everyone else. When three tests in a row come back red, a programme without a rhythm has to re-decide, every week, whether testing is still worth doing, and eventually somebody answers no. A programme with a rhythm simply holds the Friday readout, logs the three learnings, and launches the next test from the backlog, because launching is what Fridays are for. Momentum is not a mood; it is a calendar invite nobody is allowed to decline.
The backlog
A backlog item is a hypothesis, not a task: because we observed X, we believe change Y will move metric Z for segment W. Ideas without evidence attached go into a holding pen until they earn some, including the loud ones from senior people. Scoring is ICE or any of its cousins, impact, confidence and ease marked one to five; the arithmetic matters less than the arguing, because scoring is where the team is forced to say out loud why it believes something, and half the weak ideas die in that sentence.
Feed the backlog from instruments, not brainstorms: a GA4 funnel exploration shows where the money leaks, heatmaps and session recordings show why, and support tickets transcribe objections in the customer’s own words. When the well runs dry, our ranked list of test ideas exists precisely to restock it. A healthy backlog holds 30 to 50 scored ideas; ten is a programme running on fumes, and zero is next quarter’s autopsy.
Setting expectations with leadership
Programmes are killed by their own launch decks. Sell a programme on case-study winners and the first flat quarter reads as failure; sell it on the real arithmetic and the same quarter reads as the plan working. The real arithmetic: expect roughly one clear winner in four tests, expect median winning lifts in single digits, and expect the value to compound. Six winners a year at a median 4% lift compound to roughly 27% on the metric they touched, which outruns most channels the same budget could buy. Our honest benchmarks post exists to be forwarded, unedited, to whoever signs the budget. Then report quarterly in the same three numbers every time, tests launched, learnings archived and compound lift shipped, so leadership learns to read the programme’s health the way they read revenue.
The other half of the pitch is reframing losers. A losing test is not wasted spend, it is a purchase: for the cost of one build you learned that the redesign would have lost money, before it reached 100% of traffic. Teams that internalise this stop fearing red results, and teams that do not will quietly stop testing anything that might lose, which is everything worth testing.
The archive
The archive is the asset the programme is actually building. Ours holds 1640+ ecommerce experiments, and it is why a new engagement starts from priors rather than from zero; yours does the same job at any scale. Each entry records the hypothesis and its evidence, the segment and dates, the result with its confidence, the decision, and one sentence of learning a stranger could act on. Filling it in is part of the Friday readout, not homework. The archive outlives staff turnover, wins arguments against returning opinions, and stops the team from unknowingly re-running a loser from two years ago, which happens everywhere the archive does not exist.
Quarter one, week by week
The first quarter is the dangerous one, so here is the schedule we run it on:
- Weeks 1 to 2: name the owner, book the weekly slots and the week-12 review, verify analytics events against back-office orders, and build the funnel exploration.
- Weeks 3 to 4: run the research pass, heatmaps, recordings and support tickets, and write the first 20 backlog items with evidence attached.
- Week 5: score the backlog and launch the first test on the highest-traffic template, sized to read within four weeks.
- Weeks 6 to 8: hold the cadence while the test runs; launch the second test if traffic allows, and keep feeding the backlog.
- Weeks 9 to 11: read out the first results, ship or kill, archive the learnings, and launch the next tests from the top of the backlog.
- Week 12: quarterly review, velocity against plan, learnings against spend, and the decision to scale, adjust or stop.
Nothing in that schedule is clever, and that is its defence: every step is small enough to survive a bad week. A programme that reaches week 13 with an owner still in post, a cadence still running and an archive holding five honest entries has already beaten the odds that kill most of them. From there it mostly compounds, and the operating system stops feeling like process and starts feeling like the way the store makes decisions. The frame travels, too: the same owner, cadence, backlog and archive will run a personalisation effort, a speed push or an SEO sprint just as well, which is why teams that build it once rarely go back to shipping on opinion anywhere.
