| AGI might. AI? No way. See, AI was trained on existing data - on all that existing C code out there (sure, and also on all the papers and articles saying what was wrong with that C code). Those bugs are in the training data, and often not marked as bugs. So when AI generates C code, is it going to avoid making the mistakes that human code made? No, it's going to generate the kind of code it was trained on. How could it be otherwise? That's not going to nerf anything. |
| |
| ▲ | deathanatos 4 hours ago | parent | next [-] | | > See, AI was trained on existing data - on all that existing C code out there (sure, and also on all the papers and articles saying what was wrong with that C code). Those bugs are in the training data, and often not marked as bugs. So when AI generates C code, is it going to avoid making the mistakes that human code made? No, it's going to generate the kind of code it was trained on. How could it be otherwise? The generalization of this is why I think all these AI companies writing blog posts where the marketing department is just jer—ranting endlessly about how AI will improve itself into the singularity is just crazy talk. They generate a random statistically likely output, and the most statistically likely output is mid. Exceptional outputs — the ones that wow us or move the needle are exactly that, unlikely. AGI is sci-fi, and LLMs will not change that. You can see the same effect when AI emits bash, too, and especially so since most bash is terrible, and most users of bash do not put in the effort to learn bash and its foibles. So it outputs what most people write, which is not great. | | |
| ▲ | AnimalMuppet 4 hours ago | parent [-] | | It still could happen, if they had a way to judge the exceptional outputs from the mid and terrible ones. But I'm not sure they have that... | | |
| ▲ | ComputerGuru 2 hours ago | parent [-] | | In far from an AI fanatic, but I would argue training it on GitHub PRs and general software patches already provides that. Instead of just seeing the static snapshot it sees “this code was replaced by this (hopefully better) code” |
|
| |
| ▲ | CamperBob2 5 hours ago | parent | prev [-] | | When's the last time you saw a decent coding model create a buffer-overflow bug while trying to use C strings? Serious question. Anyone else seen this happen in the last 12-18 months? If so, which model and version were you using? | | |
| ▲ | smj-edison 16 minutes ago | parent | next [-] | | I use Zig, which has slices, so so far none. But man, it can't get ref counting right to save its life. There have been remarkably few times it's gotten it right on the first try. My codebase considers OOM recoverable, so it keeps forgetting to clean up memory when OOM is raised. Even in the happy path though it still messes up ref counting. I use Kimi k2.6. | |
| ▲ | smackeyacky 3 hours ago | parent | prev | next [-] | | I had Claude write a bit of stupid C# the other day that had an off by one string truncate. Surprised the hell out of me. | |
| ▲ | krupan 3 hours ago | parent | prev | next [-] | | How many people are writing C code with LLMs? I get the impression it's mostly JavaScript web apps | | | |
| ▲ | macintux 4 hours ago | parent | prev [-] | | Would you even know? Serious question. The volume of code the models can produce, the subtle ways these bugs can manifest (or even only manifest when under attack), it seems like they would be easy to overlook. | | |
| ▲ | CamperBob2 4 hours ago | parent [-] | | I have a habit of getting GPT 5.5 to review everything Opus writes for me, and vice versa. The model in the reviewer role frequently finds things I overlooked myself. Occasionally in parts of the code I wrote. No modern LLM has found any buffer overflow bugs in parts of my code that originated from another LLM. Again, though, they have found one or two that were my fault. |
|
|
|
