# Test plan

A test plan is one row per test, and a test is one hypothesis with one variable in it. The row records what you believe and why, what is being changed, what it is being compared against, how long it runs, the day the result is read, and what each outcome will mean.

The last two columns are the ones that make it a test rather than a change. A read date set in advance stops a result being read when it looks best, and a decision rule written in advance stops the finding being reverse-engineered from whatever happened.

## How to use it

1. Copy this file, or the CSV version if you would rather work in a sheet.
2. Write the hypothesis before the variant, and make sure it contains a because.
3. Name one variable. If there are two, that is two tests.
4. Set the primary metric, the minimum run and the read date before anything is built.
5. Write the decision rule for every outcome, including the one where nothing moves.
6. Leave Result blank until the read date, and then fill it in whatever it says.
7. Keep the finished rows. A year of them is worth more than any single test in it.

## The columns

**Test ID** (Text). A short stable identifier for this test. It goes on the campaign names, the creative files and the result, so a finding can be traced back to the thing that produced it.

**Hypothesis** (Text). What you believe and why, in one sentence with a because in it. A hypothesis with no reason behind it cannot be wrong in a useful way, which makes the test unable to teach anything.

**Variable** (Text). The single thing being changed: the hook, the offer, the audience, the landing page, the format. One. A test with two variables has an answer nobody can attribute.

**Control** (Text). What the variant is being compared against, named exactly. Usually what is already running. A test with no named control is a launch with an opinion attached.

**Variant** (Text). What is being tried, described precisely enough that somebody else could build it. Vagueness here is what makes a winning test impossible to repeat.

**Audience** (Text). Who sees the test, in terms the platform will accept, and identical on both sides. A difference in audience between control and variant is a second variable nobody wrote down.

**Where it runs** (Text). The platform, campaign and placement the test lives in. A test split across two surfaces is two tests, because the surfaces deliver differently.

**Primary metric** (Text). The one number this test is decided on, chosen before it starts. One. A test with three metrics will be won by whichever of the three moved, which is not a finding.

**Minimum run** (Text). The shortest period this test may run before anybody looks, written as a duration or a volume of results. It is the commitment that stops a test being stopped early.

**Read date** (Date (YYYY-MM-DD)). The day the result is read, set before the test starts. A test with no read date is read whenever somebody is curious, which is usually when it looks best.

**Decision rule** (Text). What each outcome means, written in advance: adopt, discard, or run again. A rule written afterwards is a rationalisation, and everybody in the room knows it.

**Owner** (Text). The person who answers for this test: builds it, reads it on the read date, and writes the result down whatever it says.

**Result** (Text). What happened and what was decided, written on the read date. Blank until then. A test without a recorded result is a test the organisation will run again next year.

## The test plan

| Test ID | Hypothesis | Variable | Control | Variant | Audience | Where it runs | Primary metric | Minimum run | Read date | Decision rule | Owner | Result |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
|   |   |   |   |   |   |   |   |   |   |   |   |   |
|   |   |   |   |   |   |   |   |   |   |   |   |   |
|   |   |   |   |   |   |   |   |   |   |   |   |   |
|   |   |   |   |   |   |   |   |   |   |   |   |   |

## Before you commit it

- Every row has a hypothesis with a because in it.
- Every row names exactly one variable.
- The control is named, not implied.
- The audience is identical on both sides.
- Primary metric, minimum run and read date were all set before the test started.
- The decision rule covers the outcome where nothing moves.
- Every finished row has a result written on its read date.