> LLMs are not deterministic

They are designed to be where temperature=0. Some hardware configurations are known defy that assumption, but when running on perfect hardware they most definitely are.

What you call compilers are also nondeterministic on 'faulty' hardware, so...

▲ xigoi an hour ago | parent | next [-]

While they’re technically deterministic, they’re still chaotic, in the sense that changing irrelevant details in the input (such as writing “color” versus “colour”) can make the output completely different.

▲ vlovich123 3 hours ago | parent | prev | next [-]

Even with temperature and a batch size of 1 and fixed seed LLMs should be deterministic. Of course batch size of 1 is not economical.

▲ troupo 3 hours ago | parent | prev [-]

with temperature=0 and no context. That is, a clean run with t=0, pk=0 etc. etc. will produce the same output for the same question. However if you ask the same question in the same session, output will be different.

To say the least, this is garbage compared to compilers

	▲	9rx 3 hours ago \| parent [-]
		> However if you ask the same question in the same session, output will be different. When isn't that true? `int main() { printf("Continue?\n"); }` and `int main() { printf("Continue?\n"); printf("Continue?\n"); }` do not see the compiler produce equivalent outputs and I am not sure how they ever could. They are not equivalent programs. Adding additional instructions to a program is expected to see a change in what the compiler does with the program.