This is a tautology. At some level, nobody can work at a lower level of abstraction. A programmer who knows assembly probably could not physically build the machine it runs on. A programmer who could do that probably could not smelt the metals required to make that machine. etc.

However, the specific discussion here is about delegating the work of writing to an LLM, vs abstracting the work of writing via deterministic systems like libraries, frameworks, modules, etc. It is specifically not about abstracting the work of compiling, constructing, or smelting.

▲

vidarh 5 days ago | parent [-]

This is meaningless. An LLM is also deterministic if configured to be so, and any library, framework, module can be non-deterministic if built to be. It's not a distinguishing factor.

▲

strix_varius 5 days ago | parent [-]

That isn't how LLMs work.

They are probabilistic. Running them on even different hardware yields different results. And the deltas compound the longer your context and the more tokens you're using (like when writing code).

But more importantly, always selecting the most likely token traps the LLM in loops, reduces overall quality, and is infeasible at scale.

There are reasons that literally no LLM that you use runs deterministically.

	▲	vidarh a day ago \| parent [-]
		With temperature set to zero, they are deterministic if inference is implemented with deterministic calculations. Only when you turn the temperature up they become probabilistic for a given input in that case. If you take shortcuts in implementing the inference, then sure, rounding errors may accumulate and prevent that, but that is not an issue with the models but with your choice of how to implement the inference.