LLM-is-a-compiler is indeed a simplistic approach. I wrote a rebuttal to the yesterday's Cursor post, may reuse it here https://replicated.live/blog/follow-up

The idea that a 835-page spec "just exists" and we run an LLM to implement it is completely flawed. Specs do not appear out of nowhere, they co-evolve with the code. If you have the code, why do you want to generate it again?

Good software is made as a product of numerous feedback loops and LLMs let you operate those loops faster. They do not supplement the entire process though.

In the end, a good product is a barrel of distilled feedback.

Something I have always (even pre-LLMs) found funny to think about is - all code possible to run on a computer already exists. It's some permutation of all the bits of available memory. It's in there somewhere. So, suppose you want a specific program.

1. Some huge % of those possible programs are obviously not the one you want (most don't even compile).

2. Remaining programs might look similar to the one you want, but are buggy enough to be completely unusable, unreadable to the point of intractability, and so on.

3. Remaining programs look substantially correct but upon using for > a few mins you note major bugs that make it still not correct enough.

4. Remaining programs look substantially correct and seem to generally do your will but contain a long-tail showstopper subtle bug that corrupts saved data, or makes all the output subtly incorrect, etc.

5. Remaining programs might be useful, even if they're mildly annoying.

6. This process can probably continue for several thousand iterations until you finally find "it." The program you wanted. Or... one of them. There's probably still 10k+ candidate programs left at this stage.

Our job has always been to get to 5 and aspire to 6. Indeed most of software development is just doing 5->6 in a loop.

I think LLMs help us get to somewhere between 3 and 5 faster than we used to. And a big problem with them is that programs in 3 vs 5 all already look substantially correct and there's no way to know if you're getting 3 or 5.

Generally, the above is not how it _felt_ to write software pre-LLMs, it was just a cute way to imagine what you're doing. Now it's weirdly apropos.

In my experience, a spec is often public (or available for purchase by the public from a standards committee for a few hundred dollars, close enough in business terms), but the source code for your competitor's implementations is not available for any price. The first barrier to entry to become a competitor in many industries is to implement an 835 page (note that those are rookie numbers) spec, building software that is compatible with existing data formats or legal frameworks or network protocols or hardware or whatever.

This is another example of the many invisible silos that developers end up in:

1. Some are developing enterprise software for use by their own employer, the spec only exists because it's useful to help the dev team manage the project. In this case, it's totally reasonable to expect the spec to co-evolve with the code. If the release gets pushed out to Q3 or the target has to be adjusted due to unforseen implementation difficulties, so be it.

2. Some are developing contract software for their customers. The spec exists as a communication tool and requirements that allows the developers to know what to build and to inform the customer that what they've built has reached a certain phase of the billing process (including completion). You're on a clock, and if you're late or you don't do what you said you'd do then your customer isn't going to pay you.

3. Some are developing software for abstract customers - either B2B or B2C. Often, the specs here are regulatory or industry-standard, implementing the spec is the function of the business. You're releasing updates and new products to make year-end sales targets or snipe announcements by your competition, and advertising compatibility with various standards is a crucial bullet point in the sales presentation.

Bug reports, feature requests, and especially deadlines have different meanings in each context. The way AI development agents can be used in each silo also varies enormously!

I agree with iteration, but that's an orthogonal argument to how effective it is at turning prose to code. You are asking, LLMs are capable of translating a 835-page spec, but where does the spec come from? Follow the stream up from the built software product:

Binary < IR < Code < Spec < Prompt (aka goals)

For LLMs to faithfully fulfill the role of translating spec/prompt to code, it needs enough information about the problem/motivation and the constraints around it. Often, that's done by research or by importing proven modules. The human role left in this compilation chain is a precise-enough expression of the motivation, upstream of the built artifacts.

Edit For example, SWEbench Pro was supposed to be a measure for how good LLMs are at compiling prose into code. At least 15% of those tasks were underdetermined by ambiguous prose in the spec. Of the issues that are specific enough, Claude and Codex indeed do compile to functional code with self-written tests. Source: https://github.com/kimjune01/swebench-pro-audit

Indeed, this is a much more realistic and useful framing of what to do with the current capabilities of LLMs. They work so fast, at such a high level, that you can speedrun the co-evolution process and end up with a spec and code that works. You can actually use second-system (fourth? fifth? I lost track) syndrome to your advantage because it’s so cheap to throw things away. And they don’t just write the system, they also write the test harness, the benchmarks, the failure simulation, the statistics…all the stuff that seems like “overhead” but lets you drive that evolution with data.

I’ve kind of drifted into this mode a bit at a time over the last year, but hadn’t stepped back to make such a coherent explanation of it.

As a counter argument for the compiler perspective: what Cursor did is basically transpiling a project to another language, which is a form of compilation.

If you distill the feedback into a spec (probably a combination of both docs and tests) which comprehensively covers the functional requirements, then you could treat the code as cached materialization. The value of generating code again is targeting different non-functional requirements, like ability to run in different environments or catering to different performance characteristics.

The source code always was the result of that distillation, or at least a crucial part of the result. I’m unconvinced that natural-language specs can replace that role. Even long before LLMs, when writing specs I often resort to some form of formal language to precisely nail down certain parts of the specification. And some parts are always left open, where I know I’ll get to them when writing the code, because they are more effectively solved “in the code” than waterfall-like up front.

A zen master asked a junior engineer to go get him a rock. The junior engineer asked what kind of rock he should get and the master replied by knocking him in the head with his cane. The junior thought for a minute, then asked, "would you like a round rock or a flat rock?" Again, the zen master rapped him on the head. The junior thought for a moment, then went to the creek and fetched a rock at random. He brought it to the zen master who inspected it and then said, "No, I want a flat piece of flint for starting a fire." The junior engineer was then enlightened.

Stakeholders will only give you requirements after they see the prototype. Thus, the spec and the prototype to elicit it are two sides to the same coin. Once the spec if correct and the prototype is starting to show its age, throw away the prototype and implement the spec with a fresh start.

Well, since we started training them on RLHF-style rating of the last generation they've totally collapsed back onto predictable suffix generators, which is why there are desperate, low-AUC BERT-inspired clasufiers bolted to them now.

They have like, Chomsky grammars now. And you can't fix it, it's structural to the process. You have to rewind almost to the pre-train.