Deterministic vs None-Deterministic, understanding what compiled code does also is possible without Assembly knowledge, not so necessarily with models, where I do like to at least understand what has happened and quite often find a crime or five.

Heck, OpenAI can't even get their models to reliably do what they intend, so yeah.

▲

pjmlp 11 hours ago | parent [-]

AOT without PGO, manual memory management, traditional optimisations vs AOT with PGO, JIT, tracing GC, machine learning optimisation passes.

▲

Topfi 11 hours ago | parent [-]

Which have what to do with the relationship between assembly and the higher level language?

▲

pjmlp 10 hours ago | parent [-]

It is the answer to you Deterministic vs None-Deterministic.

If you don't get the difference, I get there isn't much that you know about how instruction selection works, across all those approaches.

▲

Topfi 10 hours ago | parent [-]

Ok, I feel you know what this is actually about, but here spelled out:

Do any of the optimizations you mentioned affect the underlying behavior of the run code? Semantics...

If not, it doesn't fit to LLM use, where this is very much the case. We can talk about bugs in compilers and undefined behaviours and so on and so forth all the livelong day, but you know as well as I that this is not of the same scope of impact.

I could and in the past would have written a long diatribe about how pure compilation to assembly is deterministic, thus making this not really suitable and that while yes, there are things like JVM that aren't 100%, we have contracts, docs, specs and, because of them, an expectation of consistency that LLMs do not deliver. I did not because I thought that was not necessary, but see my error now, that is my mistake.

▲

pjmlp 10 hours ago | parent [-]

There is the observable behaviour, and the machine code instructions that actually get generated in every single case for the same kind of text input.

Which for example, unless you can exactly replicate the world every single time a PGO data gathering session is done, there are no guarantees that the compiler will generate exactly the same machine code pattern, when given various PGO files generated at different occurrences.

▲

Topfi 10 hours ago | parent [-]

Thanks, yes. To get back to the "We do still read Assembly code in the world of compilers" analogy, in the case of LLM usage, the observable behaviour is...

	▲	pjmlp 9 hours ago \| parent [-]
		Assembly naturally, https://www.phoronix.com/news/Frame-X11-Server-Assembly