These are all human-generated software bugs. At some point when AI writes all code, the bugs should all but disappear. Almost.

▲

CER10TY 6 hours ago | parent | next [-]

Bold of you to assume that AI-written and -reviewed code doesn't introduce new vulnerabilities. I think after we're done with this stretch of finding human-made vulnerabilities, we'll have a long road ahead to clean up the mess that's introduced by AI.

▲

Marha01 5 hours ago | parent [-]

> Bold of you to assume that AI-written and -reviewed code doesn't introduce new vulnerabilities.

If AI can find a vulnerability in a given piece of code, it can also do so during testing, not only when used by a malicious actor. This means that the whole thing becomes an issue only if someone is writing a safety-critical program and not doing proper LLM safety testing.

Everyone should use LLMs to test the safety of their code today.

	▲	CER10TY 4 hours ago \| parent [-]
		"Should" is doing a lot of heavy lifting there. Most shops are a long way off from secure coding, and from my experience in the last 10 years most devs lack basic awareness of security. I still see people being surprised by OWASP Top 10 or simple things like trust boundaries. There's no guarantee AI will find existing vulnerabilities proactively during coding, or that it won't introduce new vulnerabilities in the process of shipping features. You're right that shops _should_ be critical and test code, but I think due to the lack of awareness people will end up implicitly trusting any first pass written by AI and not bother to double check it.

▲

pegasus 6 hours ago | parent | prev | next [-]

The bugs will only disappear until the model that was used to security test the generated code becomes obsoleted. So, yes, this is the new cyber normal. Unless we replace the tech stack with a formally verified one. Which will need to happen, one way or another, seems to me. But cheap & sloppy generated code won't be feasible in this paradigm, so I believe we'll need a much more thoughtful approach.

	▲	thomashabets2 5 hours ago \| parent [-]
		> The bugs will only disappear until the model that was used to security test the generated code becomes obsoleted. I think you're ignoring the distinct possibility that even if the code is not formally verified, LLMs could simply stop writing bugs, or this kind of bugs. No matter how smart an attacker AI, it cannot exploit bugs that are simply not there. But hell, even formally verified ones can only verify back to the spec. There are plenty of examples of bugs in formally verified software. I place a higher likelihood on LLMs winning (producing fewer bugs than formally verified) over the long run. Mostly because verified software can, IIUC, only verify back to the spec. Whereas LLMs can go "uh, your spec itself has a bug, you really want me to codify that?".

▲

numpad0 5 hours ago | parent | prev | next [-]

Allegedly AIs sometimes even forget to implement authentication for APIs, in the way anyone can just call them. So no, bugs will continue to exist, we've just magically got tons more supply of economical labor.

▲

ninkendo 6 hours ago | parent | prev | next [-]

I think that’s a fallacy, see https://news.ycombinator.com/item?id=50207540

▲

joenada 6 hours ago | parent | prev [-]

1. The average human software-generator sucks at security. 2. LLMs are trained on human-generated software. 3. Therefore, LLMs suck at security.