  LLMs are notoriously non-deterministic, and even at temperature zero you still can't predict exactly where the weights will take you next.

An LLM can be made to be completely deterministic. I use them in this mode so I can reproduce test cases. Of course it requires complete control over the model, etc. but this myth that a computer program is non-deterministic needs to end.

You can 100% predict where the weights “will take you” given a set of inputs.

▲ shakna 8 hours ago | parent | next [-]

Floating point matrix calculations are non-deterministic. You need to invent new hardware, that doesn't use floating point math, first. [0]

[0] https://arxiv.org/html/2506.09501

▲

trollbridge 3 hours ago | parent | next [-]

Floating point math is entirely deterministic. For a given set of inputs, the same outputs come out, every time.

▲

mrob 7 hours ago | parent | prev | next [-]

>Floating point matrix calculations are non-deterministic.

This is not inherent to floating-point math. That actual (true) claim in the article is that different hardware and different hardware configurations produce different results. But deterministic inference is possible, e.g. llama.cpp on CPU is deterministic by default.

	▲	shakna 2 hours ago \| parent [-]
		Right... That's why I added "matrix" in there. Because we don't have matrix operations in software, but in hardware.

▲

feelamee 7 hours ago | parent | prev [-]

using which floating point standard? IEEE754 is totally deterministic

	▲	shakna 2 hours ago \| parent [-]
		The software standard is. GPU matrix calculations, are not. The hardware, has tiny shifts that rarely matter, except in high finance and... AI modeling.

▲ circuit10 6 hours ago | parent | prev | next [-]

I think when people say non-deterministic what they mean is closer to chaotic, like https://en.wikipedia.org/wiki/Chaos_theory as in very small changes in conditions can produce completely different output making predictions difficult

▲ Folcon 8 hours ago | parent | prev | next [-]

> You can 100% predict where the weights “will take you” given a set of inputs.

Do you mean reproduce?

Sorry it's just if you are saying what your statement implying then either the model is very simple, or you've figured out something incredible

	▲	trollbridge 3 hours ago \| parent [-]
		Sure. Take the inputs and run it through the model. You now have the output which will be perfectly reproduced with the same inputs.

▲ simonw 8 hours ago | parent | prev | next [-]

By "can't predict exactly where the weights will take you next" I meant with your brain. The blind chess analogy suggests you can predict, using your own thought process, the exact output of a prompt.

	▲	trollbridge 3 hours ago \| parent [-]
		I could use my brain to run a set weights, given enough time (on the order of millions of years).

▲ simonw 7 hours ago | parent | prev | next [-]

Can you provide steps to reproduce so I can see one of these deterministic LLMs running myself? API based or local models.

	▲	trollbridge 3 hours ago \| parent [-]
		I’d recommend starting with llamafile due to its simplicity and then move up to llamacpp and Unsloth.

▲ andai 7 hours ago | parent | prev | next [-]

Could you give some examples?

▲ techpression 8 hours ago | parent | prev [-]

You should publish, likely a Nobel price or Turing award waiting, and generational wealth at some tech giant.