Run a creative test that answers something

Two concepts that differ in the hook, the format, the offer and the landing page have told you nothing when one wins. You cannot say which difference did it, so you cannot make the next asset better — you can only make it identical.

The rule is one variable, and the test for whether you have one is whether you can write down in a single clause what you would do differently next time if this asset wins. If the clause is a sentence about a whole concept, the test is a selection exercise rather than an experiment.

Selection exercises are fine and worth running. They just do not accumulate: the tenth one teaches you as little as the first, because nothing transfers between them.

A test with no agreed end runs until somebody looks at it, which means it ends on the day the numbers happen to look interesting. Early leads in small samples reverse often enough that a test read on the day it looks decisive is close to a random draw.

Write two things down before anything starts running: the date or volume at which you will read it, and what result would count as a difference rather than as noise. The second one is what stops a two per cent gap being reported as a winner.

The corollary people dislike: if the stopping point arrives and the result is inconclusive, the finding is that it is inconclusive. That is a real outcome and it is cheaper to accept than to extend the test until it says something.

A difference you cannot detect with the volume you have is not a finding either way, and running the test anyway converts a budget into a coin flip you will act on.

The arithmetic is not complicated and it belongs before the test rather than after: how much volume each side needs before a difference of the size you care about would be distinguishable from noise. A smaller difference needs more volume, and the relationship is steep, which is why testing for a small improvement on a small account is usually not worth doing at all.

Size a test before you run it is that calculation, and the impressions-needed calculator does the front half of it on your own numbers.

The most common is a budget or bid change during the test. A large change to either is exactly the condition change a delivery system reacts to, and a test run across one has measured two things and can attribute neither.

Others: starting the two sides on different days, letting one run into a weekend the other did not, changing the landing page halfway, or letting a campaign-level budget reallocate between the two sides so that one gets most of the volume on the strength of an early reading.

That last one deserves attention because it is invisible unless you look for it. A campaign that holds the budget and distributes it between the groups underneath will reallocate towards what looks better early — which is exactly the behaviour you do not want inside a test.

Three decisions, all made before anything starts running.

Separate lines for separate jobs. A test wants its own money, not a share of a pool that reallocates. Giving it its own campaign with its own budget is what stops the allocation logic making the decision for you.

Names you can group on later. The asset name is the only field that reaches every report downstream, and it is carried onto the draft and then onto the entity. Encode the concept, the variable under test, the format and the date it entered, and nothing else. Name creatives so a report makes sense is that in full.

A written prediction. Which side you expect to win, and by roughly how much. It costs one line and it is the difference between learning something and constructing a story afterwards.

Worth being specific so the plan matches the machinery.

The workspace holds a media library of the images and videos it has and has handed to the platforms, and campaign structure reads down to ads and their creatives — so which assets are live under which groups is one question rather than a tour of several platforms.

A creative change is a change request like any other, with one detail that matters here: creative and targeting writes are complete replacements rather than field patches, which is why the undo for one is the previous specification put back exactly. Editing a live ad's creative is not a small act.

Uploading media is one of the operations no approval rule may ever clear, so it always waits for a person however the rest of the workspace is configured.

The arrangement, stated once: your connections read your accounts, a change is written as a draft on the record with the state it would change from and the checks it passed, applying it is gated on approval — by you, or by a rule you wrote — and no platform write reaches a customer account until that platform has granted write access. Applying a cleared change to a platform is in private beta.

In this order: did it get seen, did it hold attention, did it move anybody. Reading the early signals first is what turns a flat result into a next step rather than a shrug — Read hook rate and hold rate is that reading.

Then the rule that prevents the most misreadings: a ratio whose denominator is zero comes back empty rather than as a zero. An asset with no conversions has no cost per conversion. That is undefined, not free and not infinite.

And keep the loser long enough to learn from it. An asset that lost on cost per result while winning on attention is usually telling you something about what happens after the click.

The test plan template is this guide as a document you can hand to somebody, and Brief a creative for paid social is how to get assets that are testable in the first place.