Even if that's true, they are still not reliable. The same question can produce different answers each time.

hhh 2 days ago | parent | next [-]

This isn't really true when you control the stack, no? If you have all of your parameters set to be reproducible (e.g. temp 0, same seed), the output should be the same as long as everything further down the stack is the same, no?

	▲	imiric a day ago \| parent [-]
		That's not a usable workaround. In most cases it doesn't actually produce full determinism[1]. And even if it did, a certain degree of non-determinism is actually desirable. The most probable tokens might not be correct, and randomness is partly responsible for what humans interpret as "creativity". Even hallucinations are desirable in some applications (art, entertainment, etc.). [1]: https://medium.com/google-cloud/is-a-zero-temperature-determ...

▲

jowea a day ago | parent | prev [-]

Is that critical? Doesn't it just need to be better than the alternative? Unless it's a safety-critical system.