The randomness is something we add on purpose; you can set an LLM's "temperature" to 0 to get deterministic output. This tends to make the quality of its responses worse for reasons I don't think anyone really understands, but it's still functional.

I don't think the state of the art LLM providers let you do this anymore (?), but they certainly could if they wanted to, and you can do it yourself with a local model.

▲

antx an hour ago | parent [-]

Setting the temperature to 0 mathematically tells the system to always choose the absolute highest-probability word (known as "greedy decoding"), but in no reality is this "deterministic". Output drift is still a thing.

	▲	Wowfunhappy 25 minutes ago \| parent [-]
		But that's from, like, floating point errors, right? If you used higher precision that wouldn't happen, it's just because we're cheap in how we do rounding. I can see how you'd nitpick this, but to me this is a deterministic algorithm that just happens to be running on nondeterministic hardware.