I've gone back over most of my contributions to open source using LLMs and honestly, even though it was my best work at the time, there are big gaps that they find right away.

The myth that we've been shipping perfect code for years, but you can't trust LLMs, is just subjective blindness. People can't see the issues that they can't see, definitionally.

You can absolutely use the exact techniques we used to use in "the old days" to produce reliable code with LLMs generating most of it. The issue is, it's really not a lot less conceptual and intellectual effort than in "the old days", at root. You speed up the programming part, but the rest is still a hard slog, so nobody is out here doing really thorough testing in ways we used to dream of.

I think the issue people run into is that the types of errors the LLM makes are ones humans wouldn’t make, like having a basic misunderstanding of the goal of the software and making silly logic errors that technically work, but don’t serve the correct purpose.

This can be seen more clearly with self-driving cars as an example. A self-driving car may be safer than a human driver, but when the self-driving car plows into the side of a semi truck in broad daylight… that’s generally not a mistake a human would make. Humans and AI have different failure modes, so when AI fails where we generally wouldn’t, it really stands out and gets judged harshly.

Likewise, I've found many bugs using LLMs in the small software libraries I hand wrote and considered complete and correct - even though it was my sole focus at the time and I thought I had perfected it

We definitely haven't been writing completely correct code in the past. But even so, the code LLMs write has more issues than the code humans write.