Remix.run Logo
vrighter 5 days ago

if you insist that they are different, then please find one logical, non-subjective, way to distinguish between a hallucination and not-a-hallucination. Looking at the output and deciding "this is clearly wrong" does not count. No vibes.

esafak 5 days ago | parent [-]

> Looking at the output and deciding "this is clearly wrong" does not count.

You need the ground truth to be able to make that determination, so using your knowledge does count. If you press the model to answer even when it does not know, you get confabulation. What today's models lack is the ability to measure their confidence, so they know when to abstain.

player1234 3 days ago | parent | next [-]

There is no such thing as confidence regarding the actual facts, only confidence in probable output from the input. Factual confidence is impossible with current architecture.

1718627440 3 days ago | parent | prev | next [-]

And having no ground truth is what defines a hallucination.

vrighter 3 days ago | parent | prev [-]

so... vibes. got it. There is no ground truth to compare to in most cases. Because they are not in the training data where you can make objective quantifiable measurements on the statistics.