The screen does not want anything

Ask a few friends in Dubai about the adverts they see and someone will say it, half joking: my phone is listening to me. Mention a new mattress at dinner and the next morning the screen is full of mattresses. It knows. It wants me to buy.

It is a very human move. We are built to read intention into whatever is in front of us. Long ago that was the sensible thing to do. If the animal in the grass or the stranger on the road might want something, guessing what they want, as if you were them, beats waiting to find out. It pays off so regularly that we do it with everything, machines and luck included.

Sometimes it is even handy. Treating the screen as a pushy salesman with a good memory helps you predict it: look at shoes twice and shoes will follow you for a week. As a way to guess what comes next, the story is not bad.

But it is a story. There is no salesman in there, no one who noticed you, no one hoping. There is a pile of signals, some arithmetic and a great many other people who looked at mattresses too. The moment we forget that, the shortcut starts to cost us. We take it personally. We feel watched in ways we are not, and miss the ways we are. We argue with a pattern as if it could be persuaded.

The same thing is happening, more and more, with the software that talks back. It feels as though someone is in there, choosing. That feeling is useful right up to the point where the thing does something no person would, and we have no idea why.

A borrowed way of seeing is fine while it predicts well. When it stops predicting, the job is to put it down and pick up a plainer one, one that matches what the thing actually is.