> For people like me, this is just yet one more way LLMs are making my life materially worse.

I wonder if we start to hit a point where the pace that exploits are found and deployed increases faster than they get repaired, because there is a natural higher risk appetite for the bad actors, whereas the good actors still require careful thought and prudence. As a result of this, would the internet/digital economy largely seize up and crash? That would be bad, especially for a lot of people on this site. Maybe this leads to a regression in convenience, but requires more in-person interaction in order to function in society. You can already see this pattern today emerging in higher education. Maybe, in a deeper way, this improves humanity?

The coordinated disclosure thing will have to stop soon. Ideally, if you found a vuln from the defensive side, you'd tell all the "good guys" about it and none of the bad guys. The world doesn't work that way, so instead, the compromise was to tell a small number of important big actors who were unlikely to leak it, so the most good could be done before everyone found out.

Now the simple knowledge that there might be an exploitable memory issue in some given piece of software is enough for bad actors to point LLMs at it and find the issue in hours. That will only keep getting worse.

At some point, it's going to become a speed game. If you find a vuln, you might as well disclose to everyone immediately. Anyone who runs internet-exposed software will need an agent watching for disclosures around the clock and triaging them automatically (either by upgrading, improving filtering with some protective shell, or taking the whole service offline until humans can help) to keep up with the offensive agents watching around the clock and attacking automatically.

> People are saying the pace will settle down once LLMs find and fix all the bugs.

Only for projects that are in a maintenance state. New features will produce new bugs, unless everyone pays for the tokens to scan every release with the top-of-the-line agents or otherwise spends more time than many do ATM to ensure that changes and new code don't introduce significant exploit paths. Things will slow down once the low-hanging fruit from the past is mostly detected and pulled, but the level of vigilance will always have to be at least a little higher than it has been until recently.

Time between updates releasing and people installing them has been an increasing problem for some time, we are just going to have to be a lot more disciplined about that. This will be a problem for places that have strict availability requirements as they might have to compromise on the testing time they spend on new patches before updating their installations, and we might have to move to having more flexible maintenance windows (the old “an hour or two once a month after Patch Tuesday” hasn't cut it for some time and certainly won't going forward) to allow faster updates.

The bigger problem, compared to the patch-availability-to-general-use time which can be reduced somewhat via automation (though that might accelerate bad-patch based attacks…), is greater constraints for the time between bug discovery and patch release: as soon as there is a rumour of an issue many will start scanning for it (as that is easier now) so the time to wide exploitation is going to be much shorter. This might lead to buggy patches that add new issues of their own.

Perhaps we'll see a lot more projects having explicit LTS releases that only get security updates or signifiant bug fixes, which will reduce the attack surface area (and fault surface area) most users, letting those with less stability concerns deal with the bleeding edge.

> For people like me, this is just yet one more way LLMs are making my life materially worse.

I'm with you there…

> People are saying the pace will settle down once LLMs find and fix all the bugs

Yeah I don't buy it either. Look at github right now, it is a total slopfest with people vibing projects they don't know the first thing about. Plenty of LLM use will be delivering code full of new exploits. It will be interesting to see the frequency of these exploits being found correlated to LLM friendly/unfriendly contribution policies once the dust has settled to tell for certain though.

> But if you're on the side of the industry where your job is to prevent things from being broken, then it really sucks.

It's called job security :)

As an industry, we also know two more aspects

1. many have underinvested in security for a long time

2. many are checking out of the development process

I generally think we can improve software with Ai tools, but I'm skeptical the current incentives will lead to this.

I have to wonder, what is the real substantive number difference in exploits. Its in the hype train's interest to pump the CVEs in both quantity and severity so I do wonder how much of that is happening.

The time to exploit is also somewhat hypothetical. We're assuming infinite attacker resources which hasn't really been the assumption.

We see a lot more news and fear but are we seeing more actual "large scale exploitation"?

I'd love to see some numbers in that regard.

The web as we know it was never securable. LLMs made it obvious. We made a wrong turn somewhere back around JavaScript. Eventually we'll get fed up with the quick fixes and design something better. I don't blame the LLMs for this. I blame whoever made that wrong turn back in the 90's (and us, for not turning around sooner).

> For people like me, this is just yet one more way LLMs are making my life materially worse.

AI skynet is killing humans. We know that already though.

You were the one hosting an exploitable application.

> first that there is currently a much higher pace of critical vulnerabilities and exploits being discovered

The slowest that will ever be.

> second that there is a much shorter window between discovery and large-scale exploitation.

The longest that will ever be.

Aren’t exponentials beautiful?

_Is_ AI generated code notoriously terrible at security?

My experience with AI-code is that it's extremely defensive and good at minimizing 'obvious' mistakes.

Are we comparing to old models generating code? Or mythos-class models?

Can you explain the asymmetry you are claiming that exists where some models would be excellent at finding faults in code/systems and exploiting them often through coding the required tools to exploit them... but at the same time they would be bad at avoiding this kind of faults when coding the system itself?

Feels like you should be able to prompt the kind of exploits you want to prevent and the model should be able to make the system secured against them, no?

You just need to use AI to test your AI code.

(What a wonderful new world we live in)

> The rate at which we're churning out AI-generated code is going to make this worse, too, because AI-generated code is kind of notoriously terrible at security.

I think this was true in the past but AI is helping us finding exploits which have been in our code for many years. All out there by humans. We’re seeing fewer customer reported issues, and despite the pen tests getting far more rigorous, we’re passing all of them. Obviously this is one data point, but I’m hearing similar things from my friends in security and ops.

Agreed. Shouldn't continuous pen testing be a thing by now?

Yea most the people I know working in cyber security weren't worried about mythos in a serious way. All the people selling cyber products spread fear over it because profit.Everyone knows there are exploits laying in wait. Always has been always will be.

Most the cyber security people I know were upset once llms came out that could do basic os int and author their own emails, and mimic speech/video. That's how the majority of real losses occur. No one even talks about it. It's mind numbing hearing about the zebras and unicorns when there are 10 billion ants entering houses a year. Exotic stuff sells...

Now lots of sane people are genuinely and correctly worried that turning all engineers into 24/7 sloppers is and will continue to make the world far more exploitable. That's the real risk as far as I can tell

> because AI-generated code is kind of notoriously terrible at security.

and vaccines notoriously cause autism. But you might investigate why that belief is so widespread and you might not believe it that much anymore.

Can you point at something specifically that indicates this statement is true?

> The rate at which we're churning out AI-generated code is going to make this worse, too, because AI-generated code is kind of notoriously terrible at security.

So you're saying it's good at finding exploits but bad at generating code? I can tell you don't know what you're talking about because pretty much all the exploits that are being found were exploits that were enabled by human written code. So let me correct your statement for you:

There are exploits everywhere and we humans have always been bad at writing code without exploits.

> The rate at which we're churning out AI-generated code is going to make this worse, too, because AI-generated code is kind of notoriously terrible at security.

This seems incongruent with what we're witnessing. Right now we're witnessing vulnerabilities that are 6 years old be exposed. That code wasn't generated by AI slop.

Security has always been about attacker/defender symmetry. So if the attacker has an LLM so does the defender. Reality is, the tax is getting higher as security tooling extracts more perceived value.

Let’s not forget the AI industry developed the technology and infrastructure for that security nightmare to exists. And no, none of that was inevitable. At every single step the AI leadership took decisions that makes it more likely that we face the current threats

> The rate at which we're churning out AI-generated code is going to make this worse

Not only this, a significant number of developers have stopped looking at code, trusting the bots. I've seen major security regressions, bot rolled auth implementations, and falsified benchmarks... just this last week, all in projects that get hyped.

Rule #2 - do not blindly trust Ai output

https://susam.net/inverse-laws-of-robotics.html

"New normal"? Is that phrase back again? It was heavily pushed in 2020.

Walk down the street in your local neighborhood. Most modern subdivisions have a single builder, but imagine it's an older street with a variety of homes. They all stand, they all hold off the elements well enough. They are heated, they are wired, and they are certainly comfortable enough to live in.

Now imagine you are an inspector. You enter these homes one by one and discover suboptimal electrical wiring, misaligned trimming, nonsensical handle placement, lazy tile jobs, etc.

Is it a problem? Only once it becomes one. There might be 10 things in your house right now that need fixing, but you will probably not become aware of them until they reach a critical point: a leak, a fire, a collapse.

The human body is similar, no? How many ticking health time bombs are you sitting on?

Maybe this is a reminder for you to go get that exam, haha.

>of today's software

There is tons of old software still around you can use your LLM to poke at. You'll see that it was crap too. Software has always sucked, very little of it has been formally verified or built.

I remember accidentally finding buffer overflows in Netware in school but I had no idea what that actually was at the time.

As someone relatively anti-LLM, I fully agree we should run them on our own code for a bug scan. I think the only two scenarios that make sense to avoid it is if you are ethically opposed to them AND releasing something just for fun, or if you are worried about IP issues from them training on your data AND haven't released the product yet. I have something in that second category, my plan is to run an LLM on it immediately before releasing.

My experience exactly, and also Linus Torvalds experience apparently, leading to a controversial and I think misunderstood take on AI-assisted coding.

Way too many people do the parallel LLM assistance = vibe-coding, opponents call it slop, proponents claim increased productivity. But vibe coding is just one aspect, but to me, analysis is where it shines, at least when quality matters.

We have static analyzers and sanitizers, sometimes fuzzers, which I think are essential to serious development work. Now, I think we can add LLMs to the list.

We blame AI for many evils, like a lack of understanding, code ownership, poor quality output, etc... This is the complete opposite, here the AI is not there to do the thinking for you, instead it says "here look, have your thought about this?", what to do next is up to you, you are in full control of the situation. It also makes development slower instead of faster, again, the opposite of what people claim AI does. But to me, it is the good kind of slow, as in "slow and steady wins the race".

I’d like to push back on the idea that this bug would have been quite difficult for a human to discover. I’m not much of a C programmer these days, but I’ve known longjmp to be a code smell for decades now. The reasons why were broadly covered in a 1968 CACM letter to the editor by Dijkstra that we’ve all certainly heard about. Though possibly not everyone makes the connection because if all you know of the article is its title then you can perhaps be forgiven for thinking it was only talking about one language keyword in particular.

What I’m getting at is that this PHP bug was adorned with marquee lights and sirens and a giant sign saying, “Hey, look here!”

That said, when that code was written 20 years ago they could perhaps be forgiven for this approach to handling OOM. The alternatives apparently come with a pretty decent performance penalty, and culture around memory safety wasn’t as strong then as it is now. But in 2026 there is an argument to be made that exploits are so common and damage potential is so high that we need to take the same mindset that flight control software developers have in their coding standards: some ways of doing things are dangerous enough that simply using them is a defect, even when they work exactly as intended, because they are just too hard to reason about.

Alas, our brave new LLM world is making iterating one billion releases per week all too easy.

And on the positive side, LLMs make remediation much easier too. It took me around 15 minutes and a few dimes worth of DeepSeek tokens to upgrade some gnarly, janky, exploitable PHP 7.x code to the latest fully patched version of 8.3 today. Without LLMs, I would probably have been having a serious discussion about whether we should ditch the app entirely instead.

> Most software, once mature, is mostly done.

The caveat being that most software never matures. Maybe it's a cynical take, but virtually all software I use regularly is updated at least once a month. Right now, I'm using Chrome, and I see the "Relaunch to update" button waiting for me to press it. I see the arrow on the Ollama icon telling me I have to relaunch it to apply updates. I have Visual Studio Code running next to my browser, and two extensions need to be restarted to apply updates.

But at least somebody is maintaining that software.

> especially due to persistent attempts to use regexes to validate untrusted input

This is not the case here, it's an untrusted regex which can be the issue.

> sage advice on avoiding PHP

That's not sage advice. That's just biased, reactionary rhetoric. PHP is just as good or bad as anything else, and it's often the best tool for the job. As with any other language, it can be implemented correctly or incorrectly.

My concern is that while we could absolutely eliminate most security bugs through LLM-assisted scanning of source code written in deterministic languages, we are shifting away from deterministic languages because of LLMs themselves. Instead of having purpose-built programs to carry out computational tasks that have a tree of logic that you can trace out and formally prove, we're expecting a black box to guess what we want done and we can't prove what it will do.

There was a literal PoC video that started the panic, then multiple writeups. It's not hard (for people who know what they're doing) to exploit the PHP vuln. It's just not one exploit to rule every PHP installation anywhere, since this is a low-level exploit that needs to bypass ASLR, instead of a logic bug - so you need to tweak your exploit for every architecture, system, configuration, etc. And you need to know a fair bit about your target.

The response was overblown, but your dismissive reaction is too far in the other direction.

That's not true. The vuln was exploited and utilized within minutes of the disclosure.

LiveOverflow has exploited the PHP vulnerability, it’s even linked in the article.

Bold of you to assume that AI-written and -reviewed code doesn't introduce new vulnerabilities. I think after we're done with this stretch of finding human-made vulnerabilities, we'll have a long road ahead to clean up the mess that's introduced by AI.

The bugs will only disappear until the model that was used to security test the generated code becomes obsoleted. So, yes, this is the new cyber normal. Unless we replace the tech stack with a formally verified one. Which will need to happen, one way or another, seems to me. But cheap & sloppy generated code won't be feasible in this paradigm, so I believe we'll need a much more thoughtful approach.

Allegedly AIs sometimes even forget to implement authentication for APIs, in the way anyone can just call them. So no, bugs will continue to exist, we've just magically got tons more supply of economical labor.

I think that’s a fallacy, see https://news.ycombinator.com/item?id=50207540

1. The average human software-generator sucks at security. 2. LLMs are trained on human-generated software. 3. Therefore, LLMs suck at security.

The idea is that there is still an advantage even in times of LLM and the delay helps with an orchestrated fix. (E.g. not only php needed the fix, but also several docker image etc.)

But I agree it is doubtful that there is still such an advantage...

> Once the majority of code is written by similar systems then the tide of critical bugs being found will slow down.

I don’t know if this necessarily follows. I repeatedly see that AI will write code, then you have it audit the code for security vulnerabilities, and it still finds them. Just because AI can find bugs doesn’t mean it will write bug-free code. AI’s know how to take shortcuts to achieve their goals, and do it all the time.

Yes, you can have another AI audit the first AI’s code, but in practice this doesn’t always happen, especially because once the human gets their hands on the finished product, they don’t always take the extra step to ensure it’s audited. This is particularly true because people seem to think “well the AI wrote it in the first place, why would it find its own bugs?”, ignoring that context is everything, and an AI being asked to find bugs is going to approach the code a lot differently than an AI tasked with adding features.

That’s not to say it’s not possible to set things up properly into multi-agent stages with QA and security as first-class phases of development… I only mean to say that it’s not automatically true that “AI can find bugs” implies “AI doesn’t produce bugs”.

>It isn't realistic to expect new security vulnerabilities to be found at this rate

I wouldn't challenge fate like this.

+1. There aren't an infinite amount of serious security vulnerabilities out there.

> > The NHS approach: close source everything.

> Yeah good luck. I've seen how these systems work at decompiling and I expect they're probably going to be really good at fuzzing too.

Me too. The agents are much faster if it has access to source. NHS would have been easy hunting grounds. Now at least they don't stand out.

"a system should be secure, even if everything about the system—except the key—is public knowledge"

Should indeed, but we've all been caught with our pants down.