Couldn't take an article seriously that started with the author claiming that code he writes by hand is "100% correct"

Bruh

jaggederest 3 hours ago | parent | next [-]

I've gone back over most of my contributions to open source using LLMs and honestly, even though it was my best work at the time, there are big gaps that they find right away.

The myth that we've been shipping perfect code for years, but you can't trust LLMs, is just subjective blindness. People can't see the issues that they can't see, definitionally.

You can absolutely use the exact techniques we used to use in "the old days" to produce reliable code with LLMs generating most of it. The issue is, it's really not a lot less conceptual and intellectual effort than in "the old days", at root. You speed up the programming part, but the rest is still a hard slog, so nobody is out here doing really thorough testing in ways we used to dream of.

▲

al_borland an hour ago | parent | next [-]

I think the issue people run into is that the types of errors the LLM makes are ones humans wouldn’t make, like having a basic misunderstanding of the goal of the software and making silly logic errors that technically work, but don’t serve the correct purpose.

This can be seen more clearly with self-driving cars as an example. A self-driving car may be safer than a human driver, but when the self-driving car plows into the side of a semi truck in broad daylight… that’s generally not a mistake a human would make. Humans and AI have different failure modes, so when AI fails where we generally wouldn’t, it really stands out and gets judged harshly.

▲

conradludgate 2 hours ago | parent | prev | next [-]

Likewise, I've found many bugs using LLMs in the small software libraries I hand wrote and considered complete and correct - even though it was my sole focus at the time and I thought I had perfected it

	▲	ChrisMarshallNY an hour ago \| parent \| next [-]
		I have had the same experience. I’m still in the process of revisiting and refining the many hand-coded dependencies that I’ve created, over the years. Most of the issues found, were corner cases, that would likely never be encountered, but they are issues, nonetheless.
	▲	matheusmoreira 2 hours ago \| parent \| prev [-]
		Same. Frontier LLM code review proved to be a very humbling experience for me.

▲

bigstrat2003 2 hours ago | parent | prev [-]

We definitely haven't been writing completely correct code in the past. But even so, the code LLMs write has more issues than the code humans write.

	▲	ChrisMarshallNY an hour ago \| parent \| next [-]
		This seems to be changing, in the year of heavy LLM use that I’ve done. When I started, I had to review every line, and frequently found bugs, but lately, I’ve been impressed with the quality of the code. I don’t think that I’ve had to make any code-level adjustments, in a while.
	▲	jaggederest 2 hours ago \| parent \| prev [-]
		Do you have any analytic basis to make that assertion? I haven't found that to be true, it'd be more accurate in my experience to say that LLM code has different issues than the code humans write.

▲

joegibbs an hour ago | parent | prev | next [-]

I also write 100% correct code, it does exactly what I wrote it to do (not compile).

▲

an hour ago | parent | prev | next [-]

[deleted]

▲

conradludgate 2 hours ago | parent | prev [-]

she*