Remix.run Logo
▲ ashkankiani an hour ago

The confidence with which you, anonymous user, keep commenting that "hallucination is not much of a practical problem in software engineering anymore" based solely on your own anecdotal evidence is really remarkable, in not a good way.

▲user43928 an hour ago | parent [-]

You're free to substantiate your comment by telling us about your apparently different experience.

▲jazzypants an hour ago | parent | next [-]

I find hallucinations in my (mostly perfect) AI output every single day. If you're not finding them, you're just not looking hard enough. It's not surprising when everyone is screaming about how they don't read code these days.

This is just a fact. I'm sorry if it messes with your narrative.

https://arxiv.org/abs/2401.11817

▲user43928 an hour ago | parent [-]

Thanks for the link. What kind of hallucination are you seeing, and does it affect the end result?

▲ashkankiani an hour ago | parent | prev [-]

The sum total of all human observations is still not proof of the lack of hallucinations as a problem (even if their observations were perfect, which they aren't considering the volume produced vs reviewed carefully). That's why you can use a counter example only to disprove and not prove anything.

And yeah I get hallucinations all the time still. Maybe it's because I'm working on harder/more niche problems (like a compiler with an unusual type system), but it happens quite a lot. I don't record all of them.

Although the most common one you can find is them misattributing the source of changes from themselves and also other agents (Fable, Opus 5.5, deepseek, whatever). They'll say "your changes" or "you changed" or "your ruling." I didn't decide anything and it's in their own chat log, and yet...