It isn't trust

I talk about agents to my friends more than they would like. Most of them don’t work in tech, and what I show them isn’t the usual demo. Not “look, it writes poems.” How to file a complaint. How to work out which of two forms an agency actually wants from you. How to finish a thread that is seven emails deep, every one of them with attachments.

Sometimes it really gets through to them. Someone looks at it and says: wait, so it can do anything I could do at a computer? Well — yes. Roughly anything.

And then they all ask the same question. How can you trust it that much? It makes mistakes.

It does. Regularly, and sometimes infuriatingly. It confuses dates, drops a line from a form, states with complete confidence something that isn’t there. I have watched it happen and I keep watching it happen.

But I thought about that question for a long time, and at some point I realised it is the wrong question.

When people ask about trust, they are comparing the agent to some ideal version of me. The one who sits down, carefully and without errors, fills in the form, reads the contract, writes to the agency and tracks the deadline.

That person doesn’t exist.

The one who does exist looks at seventeen pages and puts it off. A month later she puts it off again. And then she does it at one in the morning, skimming, because it’s due tomorrow.

So the comparison isn’t with the ideal. It’s with what would actually have happened. And that is where it gets uncomfortable: half the things I ended up doing with AI agents I would never have done at all. Not because I couldn’t — I could. They just have too high a cost of entry and too boring a middle, and nobody sane signs up for that voluntarily.

The other half I would have done worse. Noticeably worse.

So it isn’t trust — it’s better somehow than not at all.

The gap between “done imperfectly” and “not done” is far wider than the gap between “done imperfectly” and “done perfectly.”

But the argument has a limit, and I would rather it didn’t read as “relax and trust the machine.”

It holds where a mistake is visible and cheap. Wrong number in a field — you spot it, you fix it. Misread a requirement — you go back and read the requirement.

It stops holding where the mistake is invisible or expensive. Where you can’t check the result. Where being wrong costs money, health, or your signature on something you didn’t read. There, “somehow” turns into “done wrong, and nobody noticed,” which is considerably worse than not done.

Telling those two cases apart is on you. An agent won’t do that part for you — unfortunately, and at least not yet.