I don't think there's a meaningful difference between "deterministic" and "reproducible" the way you just used them. How can something be reproducible if you can't accurately predict what it will do?

Regardless, you can make them deterministic by turning the temperature down to zero. Just nobody likes doing that for whatever reason. I guess it ruins some sort of illusion people seem to like.

IMO delegation isn't abstraction.

I work at a certain level, like Ruby code. That's what I write, debug and maintain. I don't really care about the internals of the interpreter or about the source code of Linux, because these layers are taken care of, they're reliable and they're being developed by competent people. I [think I] know what Ruby code should look like in order to remain [reasonably] fast, maintainable and reliable, and it's my job to build a product out of that kind of code. If I keep writing code like that, I know for sure that I'll be able to keep building the product, because the layers underneath are deterministic. It's like the certificate chain of trust, but with "surely these people are not idiots". And that's simply not the case with LLMs.

Not only can the LLM be a massive idiot, but also an unpredictable one. I can try to warn it, steer it, police it or review as much as you ask me to, but ultimately you're asking me to delegate my job and my responsibilities to an intermediate whose reasoning I don't understand, who has no loyalty, no sense of pride, no sense of ethics, can't be taught and can't be fired.

> It is not obvious to me that determinism is a requirement for an abstraction

But it is a requirement for them to be an "abstraction just like" assemblers or compilers, which is what was being claimed.

Writing Specs that are thousands of lines wrong without formalism, type and correctness checking is a major PITA. Wish we had a..compiler that can check our code..err I mean spec.

Writing specs validates the translation is being done properly? Was this written by an LLM?

LLMs will still blithely ignore the specs and steering documents, apologize profusely for doing so after the fact, and tell you "I'll do better next time" which they might do once or twice. But after the context is cleared or a new session opened, the Dixie Flatline gets reset and it doesn't remember it screwed up, or that you told it not to.

This happened to a coworker of mine. Generally the response from one-shotted devs is a shrug of the shoulders and "wellp, them's the breaks! As long as it looks sensible from 10,000 feet up it's still a huge productivity win." But the devil, as they say, is in the details.

> if in the future there's a prompt specific language, it would be committed. I dont think we've reached there yet, but i dont doubt this is on the path to the future

You mean, like a ... programming language? Honestly I can't tell these days what is satire and what isn't.

I posted a while ago about connecting an LLM directly to an assembler and people thought it was a bad idea. Now we're going in exactly that direction

once, if ever, the plateauing happens. Until then there is going to be this gap.

In other words, superintelligence often referred to as AGI might either be months away or just VC-Money induced cult-speak many fall victim to.

It doesn’t matter because the only certainty is that it’s not here now, and neither tomorrow etc.