I disagree with the claim that "AI agents don't get lost." What I've observed instead is that they don't experience the sensation of feeling lost. Which is quite different.

This summer I spent quite a while using a coding agent to help me untangle a deep and complicated data processing pipeline. It had itself been built by agents, in a remarkably short amount of time. But it had also become clear that it was riddled with errors and was producing lots of bad data.

What I quickly discovered was that upwards of half of my questions would receive very confidently wrong answers. And even once I had finally diagnosed whatever problem I was currently working on, it was difficult to trust the agent with any bug fixes. Since it was having an even harder time tracing data flows than I was (I'll take this chance to submit for your consideration that faster is not necessarily better), it was proving to be a bit of a monkey's paw. Yes, it would fix the exact bug I asked it to fix, but typically introduce new defects in the process. And yes, I was having this struggle with all of the latest & greatest models.

I ultimately concluded that, in this codebase, the agent was indeed deeply, hopelessly lost. (edit: And probably this code got so bad in the first place because the agents that were used to build it had been lost for a while, but unable to recognize this problem and call their operators' attention to it.)

"Confidently wrong" is definitely a standard behavior model for LLMs.

I agree with the other reply that you're likely to get better results if it has some kind of test case to run that's more authoritative than its own reasoning.

I'm curious about your exact case. In my experience I often had luck with evidence based approaches where the agent had to prove something first in order to make a statement (or write/do some tests first before making claims).

I agree though that one has to be very careful when trying to "fix" things with agents in a big codebase without it introducing new defects.

This is a really good take. Thanks for sharing this. I havent been able to put to words how I have felt about agents being untrustworthy.