Remix.run Logo
tom_ 2 hours ago

It's more likely that it isn't coincidental at all: software development-oriented LLMs became a lot better towards the end of 2025, and so there's a non-zero chance that people are using them to find new security exploits.

(People are not sleeping on this and it is not something people have failed to notice. I don't use LLMs at all and even I have noticed it - largely because there is approximately nobody that isn't talking about it.)

tptacek an hour ago | parent | next [-]

There is a 100% chance that people are using LLMs to find vulnerabilities and build exploits. If it was possible for something to be a 101% chance, that's what it would be.

tom_ 38 minutes ago | parent [-]

Apologies to all - I am British. The phrase "non-zero" does cover every case other than zero, but the intent is that it covers some cases more than others. What I'm trying to say is: yes. My intent was just to push back on this specific (and slightly bizarre to me) instance of kind-of-vagueposting, to my eyes written to imply that it might be some sort of unnoticed conspiracy, detectable only by the most enlightened of observers, attuned to the subtle signals that most people miss: that people are using LLMs to find security exploits.

tptacek 3 minutes ago | parent [-]

Right, no, what I'm snarkily saying is that basically everybody who has ever looked for a vulnerability before is now using LLMs to do it. It's a huge thing in exploit development right now.

OptionOfT an hour ago | parent | prev [-]

I think the other side is much more important. With company mandates to use AI as much as possible, there has been a deluge of low-quality PRs. Everybody is feeling tired from reviewing those, and quite possibly numerous security issues have been introduced since.

tom_ 30 minutes ago | parent | next [-]

Ahh, that's a good point, and I actually hadn't thought of that angle! I was thinking of it purely from the point of view of the attackers using LLMs to generate interesting new exploits, with a side helping of letting myself get mildly annoyed, possibly incorrectly, by the writing style.

But yes, it's also possible the defenders have been kind of forced into having the slop machine shit out a huge pile of shit-ass changes, one way or another, that end up making the attackers' job even easier. (Even assuming no mechanisation at their end! Which is of course in nearly-June of 2026, probably unrealistic. And LLMs do appear to be really quite good at that side of the equation...)

skydhash an hour ago | parent | prev [-]

The most dangerous is where the new feature works well and is using safe APIs, but integration is quietly broken somewhere. The risk of incoherent state is way higher because you no longer have a small set of people that knows the complete theory of the software and can find discrepancies.