Most testing programmes own a tool, a backlog and no documents. That is the wrong way round. The tool is the cheapest part to replace and the least of what makes a programme work, while the documents are the part that compounds, and almost nobody writes them down because none of the three feels urgent on the day it is needed.
There are three. A hypothesis written before anything is built, a QA checklist run by somebody who did not write the code, and an archive entry filled in at the readout. Between them they cover the only three moments where a test can be quietly ruined: when the idea is agreed, when the build goes live, and when the result is interpreted. We have published all three as free templates, in full on the page and downloadable as editable Word files. This is the argument for why they matter, which is the part a template on its own never carries.
The hypothesis, written before anything is built
A test with no written hypothesis does not fail loudly. It fails by producing a number nobody can act on, because the belief being tested was never stated precisely enough to be wrong. The fix is one paragraph with six slots: the evidence, the change, the effect you expect, the audience, the single primary metric, and the condition that would make you abandon the belief rather than the execution.
That last slot is the one almost every template omits and the one that does the work. A team with a real hypothesis says that if trust badges under the buy button do not lift add-to-cart rate by at least 3%, they stop treating badge placement as a lever. A team without one says it will try a different badge design, which is the same belief wearing a new outfit and a second fortnight of traffic. Both sound like learning. Only the first one can lose, and only the first one retires the six badge variations queued behind it.
Writing it costs about ninety seconds. What it buys is a developer who can build without a meeting, an analyst who knows which number grades the test, and a readout that cannot be relitigated by whoever is least happy with the answer. The full reasoning, slot by slot, is in our guide to writing an A/B test hypothesis.
The checklist, run by somebody else
A test that has not been QA’d is not a test. It is a bug with a dashboard, and it will still report a winner with a confidence figure beside it. Twenty checks in four groups cover it: the build brief and the success metric before any code is written, rendering on real browsers and real devices, event and goal tracking proved by an actual transaction, and traffic assignment watched through the first hour of live traffic.
Tracking is the group teams skip and the group that causes the most expensive failures, for a reason worth stating plainly. A broken variation looks broken and gets fixed within the hour. A broken goal produces a clean, confident report about the wrong thing, and can survive all the way to a shipping decision. Of the defects this checklist catches, tracking and goal configuration accounts for roughly 31%, ahead of responsive breakage at 24% and flicker at 18%.
One rule makes the twenty work, and it is the rule most often dropped: whoever wrote the variation does not sign it off. Self-review reliably misses the defects that follow from an assumption the author still holds, which is exactly the class of defect that survives to production. The list is also fixed on purpose. A checklist shortened for easy tests stops being a checklist and becomes a judgement call about which tests deserve care, made by the person least able to make it. It grows in one direction only: when a defect reaches production, the check that would have caught it is added permanently. Our full QA checklist walks each group in order.
The archive, which is the only one that compounds
Somewhere in your company, someone is about to re-run a test that already lost. The cost of that is not the test. It is proof the programme has no memory, and it is the failure mode nobody notices, because a programme with no memory looks exactly like a busy one.
An entry is twelve fields and takes about eight minutes when the test is fresh, which is the only moment any of it is cheap to write. Eleven of those fields are record-keeping. The twelfth is the only one that pays, and it is the one most often written badly: one sentence of learning, written for a stranger who was not in the room, and phrased as a claim about shoppers rather than a report about the test.
- Not a learning: the variant lost. It names the outcome and teaches nobody anything about your customers.
- A learning: removing the delivery estimate from the buy box costs more on mobile than the tidier layout gains. It is portable to the next template, the next redesign and the next person who suggests the same tidy-up.
- Also a learning: the effect, if real, is smaller than this store can detect in three weeks, so the next attempt needs a bigger change rather than a longer run. Inconclusive tests earn entries too, and that sentence saves somebody a month.
An archive that is only ever written to is a cost. The habit that turns it into an asset takes ten minutes and attaches to a moment that already exists: before any hypothesis is written, search it by surface and mechanism. Either the idea is already answered and comes off the backlog, or it was answered in a different context and gets rewritten to say what has changed, or nothing comes back and the search itself goes into the evidence slot. That is also why the tags have to come from a fixed vocabulary rather than free text, since three people will type urgency, scarcity and countdown for the same idea and none of them will find the others. The full case for documenting experiments covers the retrieval ritual in detail.
Three documents, one system
They are not three files that happen to sit in the same folder. Each one hands off to the next, and the handoffs are where the value is. The hypothesis names the primary metric, and the checklist verifies that exactly that metric is configured as primary before any code is written. The checklist records what was and was not verified, and the archive keeps it, so a strange number three weeks later can be traced rather than argued about. The archive answers the question the next hypothesis was about to ask, and the hypothesis carries an archive-checked line so a blank one is obvious to whoever reviews the backlog.
Break one link and the other two lose most of their value. A hypothesis with no archive behind it is a guess with good grammar. A checklist with no hypothesis has no primary metric to verify. An archive fed by tests that were never written down properly fills up with entries nobody can judge later, which is worse than an empty one, because it looks like a memory and is not.
Take them and change them
All three are readable in full on their own pages, with no address asked for, and each has an editable Word file behind a plain download link. We publish them because a template that lives only inside an article cannot be picked up and used, and because the version worth having is the one a team has already edited for its own stack rather than the one that arrived tidy.
What we would keep is the shape rather than the wording: evidence before the change, one primary metric, a named failure condition, four QA groups in that order with no shortening for easy tests, twelve archive fields and three fixed tag axes. What you should change is everything specific to you, which is the browsers your traffic actually uses, the platform you reconcile revenue against, and the surfaces your store actually has. If you would rather someone else held the pen and the build capacity, that is what experiment engineering covers, and we are easy to reach.
