Floating point matrix calculations are non-deterministic. You need to invent new hardware, that doesn't use floating point math, first. [0]

[0] https://arxiv.org/html/2506.09501

▲

mrob 8 hours ago | parent | next [-]

>Floating point matrix calculations are non-deterministic.

This is not inherent to floating-point math. That actual (true) claim in the article is that different hardware and different hardware configurations produce different results. But deterministic inference is possible, e.g. llama.cpp on CPU is deterministic by default.

	▲	shakna 2 hours ago \| parent [-]
		Right... That's why I added "matrix" in there. Because we don't have matrix operations in software, but in hardware.

▲

trollbridge 4 hours ago | parent | prev | next [-]

Floating point math is entirely deterministic. For a given set of inputs, the same outputs come out, every time.

▲

feelamee 8 hours ago | parent | prev [-]

using which floating point standard? IEEE754 is totally deterministic

	▲	shakna 3 hours ago \| parent [-]
		The software standard is. GPU matrix calculations, are not. The hardware, has tiny shifts that rarely matter, except in high finance and... AI modeling.