> how high does the bar really need to be for trusting AI agents.

You can hold a human responsible for that they do; you can reward them, fire them, sue them, etc.

You cannot do any of those things with an LLM. The threat of termination means nothing to an LLM.

Totally agree. And I expect at some point people might come around on, “don’t pay for and use that tool for that particular job.”

Like, there isn’t enough hype the world to make people replace all knives, hammers, and screwdrivers with sawzalls. They have awesome utility for certain things and they’re a bad fit for other things.

Maybe we’ll get there with LLMs someday.