Sounds like a fun project, but I have a hard time imagining it ever really catching on. I'd compare it to workflow managers. Lots of people created lots of DSLs around that, but nothing ever really caught on until Temporal, because nobody wants the cognitive overhead of needing to maintain a separate language just for workflows, especially if you don't know whether that language is going to still be around in five years.

With Temporal, you write the workflow logic in whatever language you normally use, as an ordinary async function, and if you follow the rules, it just works. Even though it looks like an ordinary procedural function, it survives server reboots, can sleep for months, etc.

I'd recommend dropping the new language approach ASAP, and shifting toward more of a Temporal-like approach. That said, Temporal does a lot under the hood and on the Temporal server side to make it worth the money. Here, I have a hard time seeing what this would provide beyond a "TReturnType LLMProxy.callAI<TReturnType>(string prompt)" function that sends the prompt and expected return type, and parses the response to the desired type. There's not even a need for a separate server tool, it's just a function.

So IDK if there's a product there or not. Seems like you'd need to figure out a way to add more intrinsic value than just a library function. But I think the new language idea, while fun to work on, is probably not going to get very far in the real world.

Sure, we cannot agree on the correct interpretation of the instructions. But, we also cannot define what is correct output.

First, the term “accuracy” is somewhat meaningless when it comes to LLMs. Anything that an LLM outputs is by definition “accurate” or “correct” from a technical point of view because it was produced by the model. The term accuracy then is not a technical or perhaps even factual term, but a sociological and cultural term, where what is right or wrong is determined by society, and even we sometimes have a hard time determining what is true or note (see: philosophy).

Yes, in general, English is non-deterministic, e.g., reading a sentence with the absence or presence of an Oxford comma.

When I programmed for a living, I found coding quite tedious and preferred to start with a mix of English and mathematics, describing what I wanted to do, and then translate that text into code. When I discovered Literate Programming, it was significantly closer to my way of thinking. Literate programming was not without its shortcomings and lacked many aspects of programming languages we have come to rely on today.

Today, when I write small to medium-sized programs, it reads mostly like a specification, and it's not much bigger than the code itself. There are instances where I need to write a sentence or brief paragraph to prompt the LLM to generate correct code, but this doesn't significantly disrupt the flow of the document.

However, if this is going to be a practical approach, we will need a deterministic system that can use English and predicate calculus to generate reproducible software.

LLMs can be run in a mostly deterministic mode (see https://docs.pytorch.org/docs/stable/notes/randomness.html for some info on running PyTorch programs).

Varying the deployment type (chip model, number of chips, batch size, ...) can also change the output due to rounding errors. See https://arxiv.org/abs/2506.09501 for some details on that.

The two parts of your statement don't go together. A list of potential output tokens and their probabilities are generated deterministically but the actual token returned is then chosen at random (weighted based on the "temperature" parameter and the probability value).

Repeatable builds are not a requirement for determinism. Since the outputs can be determined based on the exact system running the code, it is deterministic - even though the output can vary based on the system running the code.

This is to say every output can be understood by understanding the systems that produced it. There are no dice rolls required. I.e. if it builds wrongly every other Tuesday, the reason for that can be determined (there's a line of code describing this logic).

about unsolvability, it depends. In some contexts there are undecidable problems, like the halting problem, but I get your point.

yes that's my point. The article seems to suggest that the opposite is true, that without a specialist you can solve special problems with an unlearned user and a LLM.

>if the issue with deterministic generation would be solved

This can be achieved by utilizing tests. So the SWE agent will write up a set of tests as it understands the task. These are the functional requirements, which should/could be easily inspected by the BI (biological intelligence).

Once the functional requirements have been set, the SWE agent can iterate over and over again until the tests pass. At this point it doesn't really matter what the solution code looks like or how it's written, only that the functional requirements as defined via the tests are upheld. New requirements? Additional tests.

C and C++ UB enter the room,....

You seem to have missed this part of TFA:

> That means we no longer examine the code. Our time as engineers will be spent handling context, testing features, and iterating on them

IOW there would be no human to "work on, collaborate on and maintain the codebase" and so the premise of the article is that it might just as well emit machine code from the "source prompt", hence "LLM as compiler".

Or maybe you mean that this formal language is not for humans to handle but entirely dedicated to LLMs, for the sake of LLMs not having to reverse engineer assembly?

I think that's where the premises differ: the author seems to suggest that the assembly would be generated each time from the "source prompt"

I don't know, these all read like thought experiments built on hypothetical properties that these AI tools would somehow be bestowed upon in some future and not something grounded in any reality. IOW science fiction.

In the scheme described in the article the main input for AI would be the tests. If we are testing code outputs (and why not) the input then must be in a programming language.

Specifications need to be unambiguous but Natural Language is often ambiguous.

There you go. Then an actual compiler compiles the code into the correct low-level assembly for the actual linker to create an executable.

Congratulations. An LLM is not a 'compiler'.

True but this only works well if the natural language "processor" was reliable enough to properly translate business requirements into code. LLMs aren't there yet.

Exactly, and translating business requirements into code is so frustrating and error-prone that entire philosophies (and consulting firms) have been built around it. LLMs are no silver bullet, they are just faster to come up with _something_.