>LLMs, by their very nature are probabilistic.

So are compilers, but people still successfully use them. Compilers and LLMs can both be made deterministic but for performance reasons it's convenient to give up that guarantee.

▲

hn_acc1 5 days ago | parent | next [-]

AIUI, if you made an LLM deterministic, every mostly-similar prompt would return the same result (i.e. access the same training data set) and if that's wrong, the LLM is just plain broken for that example. Hacked-in "temperature" (randomness) is the only way to hopefully get a correct result - eventually.

▲

WD-42 5 days ago | parent | prev [-]

What are these non deterministic compilers I keep hearing about, honestly curious.

▲

charcircuit 5 days ago | parent | next [-]

For example looping over the files in a directory can happen in a different order depending on the order the files were created in. If you are linking a bunch of objects the order typically matters. If the compiler is implemented correctly the resulting binary should functionally be the same but the binary itself may not be exactly the same. Or even when implemented correctly you will see cases where different objects can be the one to define a duplicate symbol depending on their relative order.

	▲	ModernMech 5 days ago \| parent [-]
		That's not nondeterminism though, you've changed the input (the order of the files). Nondeterminism would be if the binary changes despite the files being in the same order. If the binary is the same holding fixed the order of the files, then the output is deterministic.

▲

PhunkyPhil 5 days ago | parent | prev [-]

GCC can use randomized branch prediction.