Data-only attacks are somewhat low-hanging fruit. Classical static analysis could already find them before AI got this strong, and LLMs make identification even easier. But the real threat is risk buried in business logic, especially abuse of normal business logic. Take e-commerce refund abuse. Bug hunters would not even call it a risk, yet fraud rings have arbitraged millions off this kind of logic. And because the logic is legitimate business logic, it is very hard to detect.

▲

eru 9 hours ago | parent | next [-]

Going on a bit of a tangent:

'Classic' non-AI fuzzers like AFL are still insanely useful and powerful, as are static analysis tools.

LLMs make all of these much, much easier to use. The other night, before I went to bed I told Kimi to go and fuzz filesystem code in the latest Linux kernel. I woke up to 26 crashes with reproducers and fixes. I'm still busy reviewing and upstreaming them. (Some have already landed.)

	▲	billypilgrim 6 hours ago \| parent \| next [-]
		Similar use case here! Combining AI with fuzzers is so powerful, especially for creating a special fuzzing harness, or generating seeds for hard to reach code. That was taking hours/days and was frustratingly boring work before. Unfortunately the Codex models refuse a lot for me, I’m mostly using the cheapest models because they refuse the least, have you found Kimi to be a good alternative? Any other you tested that you can recommend? Thinking of switching.
	▲	elzbardico 6 hours ago \| parent \| prev [-]
		It is more than LLMs are faster using those tools, than they are finding more errors by themselves analysing the code.

▲

hilariously 5 hours ago | parent | prev [-]

I still remember me and my friends on club live finding that the games you could just submit the scores for and get free xbox stuff, and then doing some research online years later we found the entire thing was setup by employees to abuse themselves with plausible deniability.

Club live lost msft millions of dollars by itself.

	▲	pixl97 2 hours ago \| parent [-]
		Lol, heh, how long before we find AI is setting up hidden doors like this to extract money from software they make via external methods.