Test plan template for Real estate

The test plan with three columns a property campaign needs: which listing or market the test runs in, which stage of the sale it runs at, and an explicit note on whether the sample can support a conclusion at all.

Everything else is the general test plan, unchanged. One hypothesis with a because in it, one variable, a named control, a minimum run, a read date set in advance and a decision rule for every outcome.

Use it across campaigns rather than inside one. A single listing rarely produces enough enquiries to decide anything, so the useful unit of testing in property is the same variable applied to many listings in one market over a quarter.

Use it again to build a record of what works in each corridor, which is the argument every listing presentation is trying to make.

Almost every property test fails for the same reason: one listing does not produce enough of anything to decide a question. A campaign might generate a modest number of enquiries and a handful of inspection attendances, spread across three weeks, in a market with its own weather.

So the listing or market column pushes the unit up. The useful test is the same variable applied across every listing in a corridor for a quarter, with the control being the way the office would otherwise have run them. That accumulates enough events to distinguish a real difference from an ordinary quiet fortnight, and it produces a finding that is about the corridor rather than about a house.

There is a practical cost: the test takes a quarter and involves several agents, which means the row has to be agreed rather than simply set up. That is why the owner column matters more here than anywhere else.

Property advertising is usually paid for by a vendor, which puts a constraint on testing that other categories do not have.

The rule that works is simple: both arms have to be versions you would have been willing to run for that vendor anyway. Testing a floorplan-led ad against an exterior-led ad is fine, because both are legitimate choices. Testing a deliberately weak creative to prove a point is not, because the cost falls on somebody who did not agree to it.

Write the rule into the plan and keep to it. It also improves the tests, because it forces both arms to be real options rather than a straw man and a favourite.

The stage of sale column stops a finding being over-applied.

A variant that wins at the open-inspection stage is winning against an audience that has already seen the property and is deciding whether to attend. A variant that wins pre-launch is winning against people who have never heard of it. The pre-auction week is different again: a small, warm audience being asked to register rather than to look.

A finding is a statement about a stage. Write the stage in the row and the finding stays usable; leave it out and the office will apply a pre-auction result to its launch creative and wonder why it stopped working.

The sample caution column is unusual and it earns its place.

Most property tests are thin. Writing down, before the test starts, that six listings is a small sample and that only a large difference should be treated as a result is what stops a modest gap being read as a finding at the read date. Everybody agrees with the caveat in advance and almost nobody accepts it afterwards, which is exactly why it goes in the row first.

It also improves test selection. A team that has to state the sample caution up front will naturally propose fewer tests and bigger ones.

Enquiry counts move easily and predict very little. A more urgent ad, a lower price guide or a bigger headline will all produce more messages, and a proportion of them are from people who were never going to attend anything.

Read on inspection attendance or registered buyers wherever the office's system records them, and use the system's own words. Where neither is recorded reliably, fixing that is a better investment than any test in the plan, because without it no finding in this category can be trusted.

Property markets have strong seasons, school terms, holiday periods and weather, and all of them move inspection attendance by more than most creative variables do.

Run both arms simultaneously across the same listings and weeks rather than sequentially, so the season moves under both. Where the test spans a seasonal boundary, note it. And treat any result read across a holiday period with particular suspicion, because attendance in those weeks is dominated by who was in town.

The completed rows are worth more than any single test, and in this category they are worth more than usual, because they answer the question every listing presentation tries to answer: what works in this corridor, at this bracket, for this kind of property.

Store them by market. After a year an office has its own evidence about first images, price guide presentation, inspection timing and creative formats in the streets it actually works in, which is more persuasive to a vendor than anything general.

Every row has a hypothesis with a because in it. Every row names whether it runs on one listing or across a market, and most should name a market. Every row names the stage of sale. The sample caution is written honestly before the test starts. Both arms are versions you would have been willing to run for the vendor anyway. The primary metric is an inspection attended or a registered buyer. And the decision rule covers the outcome where nothing moves, which with these volumes will be common.

A test that passes those seven is not guaranteed to produce an answer. It is guaranteed not to produce a false one, which in a category this thin is the more valuable of the two.