A test to settle an argument

A test is supposed to be a question. Most of the ones I have watched were an answer somebody already wanted.

Two people disagree. One thinks the offer is the problem, the other thinks the creative is. Neither can win the meeting, so somebody proposes testing it, and the room relaxes, because a test defers a decision without looking like deferral.

Then watch how it gets built.

The senior person's version takes the larger share of the budget, because it is the one the plan already assumes. It gets the placement everybody knows works. It gets the audience that has bought before. And it gets read on the third day, while the numbers are still moving, which is exactly when a favourite is ahead.

None of that is dishonest. It is what happens when the person designing the test is also the person who has already said what the answer is.

I have seen enough of this from the platform side to look at the split before the result. An even split is a question. An uneven one is a plan with a control group attached, and the control group is there so the plan can be described as tested.

So the useful move is uncomfortable and free. Give the design to the person who expects to lose.

They are the only one in the room with a reason to make it fair. They will ask for the same budget, the same placement, the same audience, and a reading day chosen before anything runs, because those are the only conditions under which their side has a chance. The senior person still owns the decision. They stop holding the pen that decides what counts as evidence.

A test designed by the person who wants to win is not an experiment. It is a defence, prepared in advance.