Why AI Agent Are Winning Benchmarks and Losing Users

A few months back I picked up a book called Alchemy by Rory Sutherland. He is the vice chairman of Ogilvy, the advertising firm, and he has spent thirty years watching how people actually make decisions versus how engineers and economists think they make decisions. Spoiler: those two things have almost nothing in common.

The whole book is built around one idea. People are not rational. They don't pick the best product. They don't respond to the most logical argument. They buy what feels right. What signals something. What fits the story they are already telling themselves.

He calls it psycho-logic. The human mind does not run on logic any more than a horse runs on petrol.

Here is why that matters right now. And why almost nobody building AI agents seems to have gotten the memo.

The AI Agent World Is Built Entirely on Logic

Go look at how AI agents are marketed and compared. Every benchmark is about capability. Token limits. Context windows. Reasoning scores. Speed. Cost per API call. The builder community follows the same track. Did it complete the task? Did it hallucinate? How many steps did it take?

These are all logical questions. And if Sutherland is right, and I think he is, the entire industry is solving the wrong problem.

Because here is what actually determines whether someone uses an AI agent or not. Whether they TRUST it. Whether it makes them feel capable rather than replaced. Whether the experience of using it feels right, not just whether the output technically is right.

None of those things show up on a benchmark.

Sutherland uses Red Bull as one of his favorite examples. By every logical measure it should have failed. Expensive. Small ugly can. Tastes bad. A logical analysis of the beverage market would have killed it before it launched. But Red Bull tapped into something psychological. The small can signals concentration and potency. The bad taste signals that it is medicine, that it is working. None of this is logical. ALL of it is real.

The AI agent equivalent would be an agent that is not necessarily the top benchmark performer but makes the user feel like a superhero when they use it. One that builds confidence instead of anxiety. One where the interaction feels powerful, not just the output.

AI companies are not building those. AI companies are building agents that win on paper and confuse people in practice.

The Black Box Problem

Here is what the existing frameworks actually feel like to use. You hand something off. The machine disappears into its own reasoning. Something comes back. You have no visibility into what happened, no control over intermediate steps, and no ability to course correct until the whole thing is done.

That is not a technical problem. That is a trust problem. And it is exactly why adoption stalls even when the output is technically good.

Most agents dump everything into a wall of output. They show all their reasoning. They surface every intermediate step. Logically this is transparency. In practice it creates cognitive overload and anxiety. Users see the machine thinking out loud and instead of feeling informed, they feel uncertain. A Nielsen Norman Group study found that 72% of users said the language and behavior of an AI directly impacted their trust in it. Not the accuracy. Not the speed. The EXPERIENCE.

ICM Gets the Psychology Right Without Trying To

This is where ICM gets interesting in a way that has nothing to do with its architecture.

The existing frameworks built for engineering elegance accidentally created the black box problem Sutherland would predict kills adoption. ICM built for simplicity and human control accidentally solved the trust problem without ever framing it that way. The human is not watching a machine work. The human is working with a machine.

That is a completely different psychological experience and it maps directly onto what Sutherland is talking about. Every step is visible. Every output is something you can touch and change before the next stage runs. The checkpoint before each stage creates the feeling of partnership instead of delegation into the void.

The builders using heavy frameworks are solving a logical problem. ICM is solving a psychological one.

Sutherland has been saying for thirty years that the psychological solution usually wins. The most capable agent in the room is not always the one people use. It is the one that makes people feel good about using it.

That is the whole game.

1 comment