I am an educator alongside being an engineer, so I've had to think about how to explain this topic to people in ways that give them some kind of intuition/insight. I don't have a good take for non-stem people, but I think I have a better explanation for people who are CS adjacent

I like to explain this whole hallucination problem by stating that LLMs are 2 different machines working together. one half of the machine is all the knowledge it was trained on, and you can think of this knowledge as an enormous classic tree you learn in CS classes; and each node in this tree is a token. the other half of the machine is a program that walks through this enormous tree and prints the token it's on

when you think of it like this, 3 things become immediately obvious

1. LLMs are a totally deterministic machine

2. you can make them seem smart by randomizing the walk through the knowledge tree

3. hallucinations are a side effect of trying to randomize the knowledge tree walk

I find it interesting that LLM companies are trying to fix such a fundamental problem by training the model to always guess the correct path. the problem I see with this approach is that 2 people can enter the same input text, but want 2 different outputs. if there isn't always a _correct path_ then you can't really fix the problem.

the only 2 options you have to “improve” things is prune and or add better data to the knowledge tree, or you’re trying the make the program that walks the knowledge tree take better paths.

the prune/add data approach is slightly better because it’s improving the quality of the token output. but the downside is you quickly realize that you need a fire hose of new human data to keep improving - but much of the data out there is starting to be generated by the LLMs - which leads to this inbreeding effect where the model gets worse

the 2nd approach feels less ideal because it will slow down the process of generating tokens.

all of this to say, from this point on, it’s just hacks, ducktape, and bandaids