When people refute the idea that "LLM is the new compiler from natural language to code" the common reason is that LLM is non-deterministic but compiler is deterministic.

But the determinism isn't the real distinction. LLM inference can be made deterministic (temparaute zero, etc.). And multi-threaded compiler can be non-dterministic.

And human developers are also non-deterministic. There is always the nondeterminism even before AI.

I think the real difference is how leaky the abstraction is. If you just need to understand language instead of compiler internal to write correct code, the compiler is non-leaky abstrction. If you rely on compiler to do auto SIMD optimization, and have to do hacks to trigger the optimization, compiler is leaky abstraction.

For AI, if you don't ever need to read the code and AI writes perfectly bug-free program that perfectly follows instruction, then AI is a non-leaky abstraction, but that's not the case for now. To make code maintainable and debuggable you still need to read the code if the project is complex enough.

> And human developers are also non-deterministic. There is always the nondeterminism even before AI.

Pointing out humans are not deterministic is irrelevant to the comparison. In the original example of LLMs replacing compilers, the deterministic compiler is being replaced, not the non-deterministic human. In the original example, we move from non-deterministic human -> sourced code -> deterministic compiler, to non-deterministic human -> sourced prompt(?) -> non-deterministic LLM. The important part is the sourced code into a deterministic final step so your build artifact doesn't change.

You can point out you can change temperature to zero or what ever, but then you don't actually get the build artifact you wanted in the first place.

Yeah, it's not the non-determinism; it's that they don't do what I say. I can't imagine writing a nontrivial program without, at a minimum, reading and editing the pieces as they're being produced. People aren't complaining about non-determinism; they're complaining about the results being non-deterministically sometimes wrong.

> And human developers are also non-deterministic. There is always the nondeterminism even before AI.

Humans are not machines. The entire point of any technology (including specifications derived from consensus, such as the metric system, democracy, etc) is that it operates deterministically in the service of (as opposed to outside of) actual people with actual needs. This is what makes a tool a tool, and not a human. To me, it all sort of boils down to the shortest path between an intention and it's result. LLM generated code can massively shorten the time between A and B, but not necessarily shorten the path.

The feature that makes LLMs great and interesting is the same feature that makes them hallucinate. This is embedded in the entire idea of coherence being the byproduct of nudging weights in proximity as opposed to being constructed toward an unknown future. I thought it could be cool too, but a deterministic AI (zero temperature, frozen model, or whatever..) would _only_ really serve to be a leaky abstraction over its training sources. Which maybe says more about our access to information and desire to understand it.

I don't think there's a clear cut line (or even value in one!) between LLM output being good or not, but without recognizing that LLM output will always be riffing on prior inputs as opposed to being intentionally constructed toward an unknown future, we've probably lost the plot.

" if you don't ever need to read the code and AI writes perfectly bug-free program that perfectly follows instruction, then AI is a non-leaky abstraction, but that's not the case for now."

? it can never be the case.

AI has a 'noisy channel problem' and it's due to the ambiguity inherent in language.

If we have a 'way to express ourselves perfectly' in a language-like concept, at a higher level of abstraction than code, we'd have our solution.

But we don't.

We could develop really, really strong ways of writing specifications that work really well, combined with other kinds of paramterized specs, that could work - but as of now, we're not quite there.

We are 'missing a layer' - and do we make do with what we used before.

> I think the real difference is how leaky the abstraction is. If you just need to understand language instead of compiler internal to write correct code, the compiler is non-leaky abstrction. If you rely on compiler to do auto SIMD optimization, and have to do hacks to trigger the optimization, compiler is leaky abstraction.

Autovectorization is not a programming model!

I forgot who put me on that article - I think it might've been Horace He - but that is a pretty big idea in idea space for me.

I phrased your view differently in my earlier comments - we already allow "nondeterminism" (or better call it chaotic behavior) in compilers. But usually it is only limited to performance (CPU and memory consumption).

So the "LLMs are chaotic" (difficult to predict) argument is somewhat moot; the performance of GC language runtime or even modern CPU with hyperthreading is already chaotic too.

I think for many applications, people will treat correctness the same way.

> But the determinism isn't the real distinction. LLM inference can be made deterministic (temparaute zero, etc.). And multi-threaded compiler can be non-dterministic.

And yet, I've never used a compiler that had non-determinism I couldn't turn off, or an LLM product where Icould turn it off. I don't see the theoretical possibility for the opposite behavior as relevant if there are approximately zero examples in practice. In the real world, you can assume your compiler is deterministic and your LLM isn't, and you'll almost never be wrong. That's a real difference that abstraction is not going to fix.

>But the determinism isn't the real distinction. LLM inference can be made deterministic (temparaute zero, etc.).

You'll still be exposed to non-deterministic behaviour through floating point errors, GPU batching and a myriad of other factors.

I don't think it is abstracting over code. It's abstracting over people. Let's do a concrete example instead of philosophizing.

"Build me a web dashboard where I see all my OSs processes etc. Maybe also memory usage and whatever, think of cool stuf. Ping me when you're done." is a perfectly valid instruction. It's vague, it's not code, but it is a perfectly valid instruction. In fact, I regularly deal with instructions from my boss that are orders of magnitude more vague and unclear than this.

What happens then is engineers scramble, gather their thoughts, present their ideas to the pointy-haired-boss and inquire about the context, the Why, the desired end state, the planning, etc. Given that "programming" is slow, extraordinarily costly and error-prone we have to carefully manage this. We create processes, systems of accountability and governance. After a "sprint" or two or ten some sort of dashboard will appear that will have about 60% of what was actually required if we are lucky (because the rest was unstated, heavily implied or genuinely unknown or some unholy combination of those things). We will have more meetings. Some will consist of stylistic discussions, some will discuss the "framework" and some "deployment". At least one will detail either the merits of a certain set of icons, the beauty of htmx or whether we should use Rust or Go on the backend. At least one person will argue for Elixir.

AI agents turn all of this into a "here is a result" and you'll be learning new requirements and iterating constructively on the idea itself instead of wasting time on NixOS and the nitty gritty of Material Design. You want Elixir? Ok sure.. let's do that in five minutes instead of twelve weeks.

I'm not at all saying the "nitty gritty" is useless or is a complete waste of time _generally_. I think I am saying that it _usually_ is a waste of time, in my bubble, and many projects would be served well with quick, concrete iterations on the ideas themselves instead of getting bogged down in bullshit. This is of course just my experience. I admittedly live in a weird bubble.

Temperature zero doesn’t mean the LLM becomes deterministic

https://mbrenndoerfer.com/writing/why-llms-are-not-determini...

[dead]