>> an LLM is completely opaque

And despite that, although they are not like that in practice as there are too many uncontrolled variables, with temperature at zero, for the same input they produce always the same reply.

▲

irishcoffee 4 hours ago | parent | next [-]

Ha, they sure don’t.

	▲	Betelbuddy 2 hours ago \| parent [-]
		They do. Just train your own LLM, not that difficult, and you will have a more controlled environment and you will see they do.

▲

chrisjj 3 hours ago | parent | prev [-]

> with temperature at zero, for the same input they produce always the same reply.

Nonsense.

https://thinkingmachines.ai/blog/defeating-nondeterminism-in...

▲

coldtea 2 hours ago | parent [-]

BS. Run them sequentially on a single core, and without fancy speedups enabled, and they do. The algorithm is determinstic. Any non-determinism present with 0 temperature it's not some mysterious LLM-inherent property, but something that can be seen in any large program taking advantage of multi-core, floating point, and other CPU-based parallelism optimization.

	▲	hashstring an hour ago \| parent [-]
		Exactly, this is correct. People often assume they are not because they can ask the same query to the same model and get differences in output, but wrongly conclude that this is some inherent LLM trait, instead of non-determinism added on top of it because of implementational choices that were made.