You’re technically correct, but the vast majority of software has never been built to the kinds of standards you are describing. LLMs are not displacing that kind of work!

>“Finding out that it doesn't” means that you didn’t properly reason through the code beforehand, checking all your assumptions against what the code and underlying systems are actually guaranteeing. This may be a matter of formal education (proving computer science theorems and algorithmic correctness in university), I don’t know.

We're not writing theorems, dude.

Except in the equally pedantic sense that every program is a proof to a theorem...

We're writing plain enterprise and web software, closer to CRUD than NASA.

If you said that even before LLMs 0.1% of teams "checked all assumptions against what the code and underlying systems are actually guaranteeing" in any kind of formal way, you'd be overestimating it.

Your code is only as good as what you can prove. Understanding the code is not the goal, it’s only important insofar as it helps you evolve the codebase predictably and without bugs or regressions, and understanding is not easily measurable or transferable.

Moreover, when your codebase is hundreds of thousands to millions LOC, I question how much you can ever truly understand it at the level you’re saying.

> believing you can understand the behaviour of a program without at least reading the high level code is truly silly.

have you ever used a library after only reading the README and documentation, or do you always pull the source and read through it before you think you understand it?

Before 2024, I once went to an ATM to retrieve money and selected 50. Note that I selected it from a menu, not typed it. The ATM then told me that it cannot give me 50 because it is not a multiple of 5.

Yeah no one wrote buggy code before 2024 right

I mean, I think the problem isn't that the LLM doesn't know how to code, it's that companies are expecting 3-5x velocity with the bottleneck of code review and testing becoming much more severe than before

if you're an MBA-brained exec who doesn't actively use LLMs to code and you just believe whatever slop it outputs at first without checking it, you're not going to realize how recklessly it can be used, how you need to be critical and skeptical of its outputs, that you need to explore it's reasoning and logic (which is still really easy compared to understanding legacy code and barely takes any time!)

say you also believe all this marketing hype about 'how dangerous (ie capable) AI agents are.' LLMs can do anything you think so you just say 'ship it' without building out the tooling and capabilities to enable faster code review and better tests. and to keep the shareholders happy, you start cutting jobs that you can't directly connect to a KPI (ie the platform/SRE team who would be the ones who can trial, onboard, and maintain those capabilities for your teams)

and from this, suddenly a lot of debit card stops working and the only one getting the blame are individual SWEs trying to hit their sprint velocity. the fact that you fucked up the whole SDLC real bad with your incompetence gets you a golden parachute and you job hop to a better paycheck. rinse and repeat

Debit card transactions still work

> Your debit card transactions for example worked.

I've built payment rails. Six nines SLA, high capacity, resilient distributed systems.

I haven't written a single line of code since February, and I don't think I ever will again. These systems are incredibly good at replacing much of our work. They're only going to get better.

Rather than debating if these models are good (they are), we should be trying to figure out if most of us will still be around in three years. You don't need a two pizza team anymore.

"Look to the person to your left and to your right. Only one of you will remain by graduation" kind of energy. I'm not sure all of us is going to be in this career much longer. We'll have to see what the demand side looks like.

Yes, but Have You Tried the Latest Model?

/s

Why or how is software worse now? From my perspective we are entering a golden age of software, cheaper, better, faster.

> Strange that the world worked before 2024

It must have been a huge shock when you were suddenly transported from a working parallel universe into ours back in 2024.

If you can quantify quality/reliability/understandability, you can tell LLM what kind of code do you expect. If not, you get whatever.

At my current place we not only have automated tests, static analysis and static rector (linting, but also automatic pattern matcher for problematic code) but also: - architecture tests that define relationships between application layers - ADRs that guide developers (and agents as well) that communicate how new code should be written and how existing code should be treated

I find that "how code should look like"/"what code should do" is an ambiguous idea that always is preached, but never defined = everyone's idea of quality is slightly different and only looking at existing code you tend to align. Everyone's idea of what the product does/should do is kept within their heads. If we define this knowledge in writing LLMs can not only write code according to the patterns that are thus defined, review existing code based on these documents, but also actually read acceptance criteria documents to check if the code does what it's intended to do (gherkin)

Same goes for understandability - if LLM applies one pattern this time, another pattern another time, if you have multiple coding patterns then that hurts clarity. Sometimes LLMs work as common denominator thus achieving clarity, but I find that actually giving LLMs reference works.

> I agree, I think this false dichotomy between using LLMs and caring about quality/reliability needs to stop

You're not going to get people to stop doing that by arguing on the internet, but in the end it won't matter, because it will stop, naturally.

In the future, you'll just get left behind and not hired if you're building code by hand, it's that simple. Even traditional code reviews are going to go away. It'll be more about the scope and then verifying correctness.

> I agree, I think this false dichotomy between using LLMs and caring about quality/reliability needs to stop. All these mission critical industries listed in the article rely on extensive testing for quality assurance, with human code review being a layer on top of all that, but far from the most critical one.

At least some places are abolishing formal QA because LLMs. There's a cult of speed uber alles that has a big intersection with LLM enthusiasm.

>TFA's points only make sense if the mental model the author has in mind is someone who writes a prompt then immediately puts an app into production without any thought behind it.

If coding were solved, then this would be true no?

> I think this false dichotomy between using LLMs and caring about quality/reliability needs to stop

I agree. What does coverage-guided fuzzing fuzz if there is 100% test coverage?

So, then, 100% branch test coverage is not a sufficient metric (because it doesn't indicate whether the code is fuzzed or formally verified for example).

Would Branch coverage even be a sufficient software quality metric if we were to instead measure how many times each branch of code is covered by tests? How to verify that one test which executes 100% of the code and runs only one assertion on, say, a CLI utility exit code integer is actually sufficiently covering?

> I think a codebase generated by AI is actually more understandable than one generated by humans at this point,

From doing a larger port (of sphinx, docutils, myst-md-parser, pygments, to rust in westurner/dsport) with a lot of human in the loop and currently ~80% branch coverage, this seems to be at least initially true but just like real life there's drift from even a good plan that you pay a more expensive model to prepare.

I suppose it's the same challenge as architectural drift in open source non-LLM-assisted products and the solutions are pretty much the same: give better instructions (AGENTS.md,) and use better sufficiency criteria as an engineering manager (branch test coverage, fuzzing, formal methods, TLA+), and train and pay humans to do secure code review.

Sometimes the agent doesn't notice that the code already solves for that and implements its own implementation with tests and it's wastefully redundant when the code should be refactored and the tests should be refactored so that we can delete code in order to minimize bloat.

Unfortunately often, just like IRL software development, the response from the agent is not sufficient to close the issue.

One proposed solution for this that is in retrospect obvious and also essential to success in "normal"/"traditional"/"legacy" (non-AI) engineering projects, is to always verify whether the candidate solution satisfies the criteria;

From "Groundtruth – checks your AI coding agent's claims against the Git diff" https://news.ycombinator.com/item?id=48838209 :

> "Follow up to verify that the work was actually satisfactorily completed"

> Are there other sound management practices that aren't yet effectively implemented in current gen agents?

Oh, and always write tests, docs, commit messages, and changelog entries; but don't waste tokens on documenting something that doesn't verifiably pass sufficient tests.

> I agree, I think this false dichotomy between using LLMs and caring about quality/reliability needs to stop.

It's not just a false dichotomy, it's intellectual dishonesty. It wasn't that long that conversations about code quality, technical debt, etc were on the front page of HN on the regular. Whether it was coding bootcamp grads who had just enough confidence to be dangerous, "just ship it!" cargo culters, or the product of management breathing down the necks of otherwise good developers, there's plenty of "human slop" running in production across servers worldwide.

It would be deliciously ironic if AI was the straw that broke the camel's back where a deluge of bugs and anti-AI sentiment caused the public to vote for legal liability for software defects and licensure of developers. No more of this "no warranty, express or implied" business for us.

I'm rather surprised to hear this. This feels like a post from about 18 months ago. I can't remember the last time I encountered a genuine code hallucination from a frontier model. They have other issues, but rarely this.

What kind of domain are you working in?

> "out of order"

Those who bring up OoO as if it caused unpredictable execution should ask their LLM what a reorder buffer is and why it exists.

Article 'the' is doing a lot of work. I read it as author understands code but doesn't understand THE code.

I came here to say something similar. We won’t need to understand the implementation. But we will need to understand the requirements. The tests, or some higher level DSL they’re (deterministically! not via LLMs) generated from, will still need to be human verified.

They aren't making that claim. They are saying AI can improve on and supplement error-checking. And some error-checking will in fact be made redundant by this tool - but obviously not all of it.

Thank you!! I saw this video about a decade ago, couldn't remember where it came from, and was having trouble finding it again! Bret Victor is an HCI juggernaut. From this info, here are some other links:

his website: https://worrydream.com/

his page for learnable programming: https://worrydream.com/LearnableProgramming/

There was a programming environment closely related to his work that was a kind of visual database. But it may not have been developed by him / his lab. I don't see any obvious links to it on his website.

Thank you for sharing :) good food for thought