Remix.run Logo
▲ catlifeonmars an hour ago

I think OP is saying that hallucination or not is just semantics. There is nothing qualitatively different about hallucinated vs non-hallucinated output.

▲bonoboTP 30 minutes ago | parent [-]

That's true in the same sense as "There is nothing qualitatively different about erroneous vs non-erroneous output" for a dog vs. cat image classifier.

▲user43928 22 minutes ago | parent [-]

To be fair, I guess the line is blurry between what could be labelled a regular mistake compared to a hallucination.

"Test suite passed" when it actually errored? Obvious hallucination, unless it ran a command that returned the wrong error code.

But is running a malformed command that does not achieve the expected effect itself a hallucination?