Cheap traffic, wrong answers

Every August somebody suggests using the quiet month to run the tests. The traffic is cheap, the stakes are low, and the team has time for once. It sounds like the most sensible fortnight of the year.

It is the worst month to learn anything, and the cost does not show up in the test budget.

Look at who is actually in the market. Half the city has gone. The people still ordering are the ones who did not travel, or could not, or came back early, and they are not a smaller version of the customer base. They are a different one: different incomes, different reasons for buying, different baskets, different hours. A test run in August answers a question about them, precisely and honestly.

Then September arrives, the city comes back, and the answer gets applied to everybody.

The headline that won did not win because it was the better headline. It won because it spoke to whoever was left. The price point that worked worked on a thinner wallet, or a bored one. The landing page that converted converted for people with nothing else on. None of that is written on the result, which arrives with a confidence interval and no note about the month it came from.

From the platform side, the accounts that spent August testing came back in September certain and wrong, which is a worse position than coming back uncertain. Certainty spends faster.

There is plenty worth doing in a thin month. Fix the tracking nobody has trusted since March. Rebuild the creative library. Write the September offer properly. Clear the small jobs that were never anyone's.

August is a good month to build. It is a bad month to conclude.