> LLMs hallucinate because training on source material is a lossy process and bigger,

LLMs hallucinate because they are probabilistic by nature not because the source material is lossy or too big. They are literally designed to create some level of "randomness" https://thinkingmachines.ai/blog/defeating-nondeterminism-in...

▲

ChadNauseam 6 hours ago | parent [-]

So if you set temperature=0 and run the LLM serially (making it deterministic) it would stop hallucinating? I don't think so. I would guess that the nondeterminism issues mentioned in the article are not at all a primary cause of hallucinations.

▲

joquarky 6 hours ago | parent [-]

I thought that temperature can never actually be zero or it creates a division problem or something similar.

I'm no ML or math expert, just repeating what I've heard.

	▲	ChadNauseam 5 hours ago \| parent [-]
		That's an implementation detail I believe. But what I meant was just greedy decoding (picking the token with the highest logit in the LLM output), which can be implemented very easily