Remix.run Logo
9cb14c1ec0 3 hours ago

Now, its very possible that this is Anthropic marketing puffery, but even if it is half true it still represents an incredible advancement in hunting vulnerabilities.

It will be interesting to see where this goes. If its actually this good, and Apple and Google apply it to their mobile OS codebases, it could wipe out the commercial spyware industry, forcing them to rely more on hacking humans rather than hacking mobile OSes. My assumption has been for years that companies like NSO Group have had automated bug hunting software that recognizes vulnerable code areas. Maybe this will level the playing field in that regard.

It could also totally reshape military sigint in similar ways.

Who knows, maybe the sealing off of memory vulns for good will inspire whole new classes of vulnerabilities that we currently don't know anything about.

woeirua 2 hours ago | parent | next [-]

You should watch this talk by Nicholas Carlini (security researcher at Anthropic). Everything in the talk was done with Opus 4.6: https://www.youtube.com/watch?v=1sd26pWhfmg

fintech_eng an hour ago | parent | next [-]

its also very easy to reproduce. i have more findings than i know what to do with

redfloatplane 2 hours ago | parent | prev [-]

Thanks for sharing that talk, enjoyed watching it!

Gigachad 24 minutes ago | parent | prev | next [-]

Apple has already largely crushed hacking with memory tagging on the iPhone 17 and lockdown mode. Architectural changes, safer languages, and sandboxing have done more for security than just fixing bugs when you find them.

georgemcbay 2 hours ago | parent | prev [-]

> It will be interesting to see where this goes. If its actually this good, and Apple and Google apply it to their mobile OS codebases, it could wipe out the commercial spyware industry, forcing them to rely more on hacking humans rather than hacking mobile OSes.

It will likely cause some interesting tensions with government as well.

eg. Apple's official stance per their 2016 customer letter is no backdoors:

https://www.apple.com/customer-letter/

Will they be allowed to maintain that stance in a world where all the non-intentional backdoors are closed? The reason the FBI backed off in 2016 is because they realized they didn't need Apple's help:

https://en.wikipedia.org/wiki/Apple%E2%80%93FBI_encryption_d...

What happens when that is no longer true, especially in today's political climate?

tptacek 2 hours ago | parent | next [-]

Big open question what this will do to CNE vendors, who tend to recruit from the most talented vuln/exploit developer cohort. There's lots of interesting dynamics here; for instance, a lot of people's intuitions about how these groups operate (ie, that the USG "stockpiles" zero-days from them) weren't ever real. But maybe they become real now that maintenance prices will plummet. Who knows?

qingcharles an hour ago | parent | prev | next [-]

I assume that right now some of the biggest spenders on tokens at Anthropic are state intelligence communities who are burning up GPU cycles on Android, Chromium, WebKit code bases etc trying to find exploits.

fsflover 2 hours ago | parent | prev [-]

> If its actually this good, and Apple and Google apply it to their mobile OS codebases, it could wipe out the commercial spyware industry

If Apple and Google actually cared about security of their users, they would remove a ton of obvious malware from their app stores. Instead, they tighten their walled garden pretending that it's for your security.

fsflover 2 hours ago | parent [-]

Some links for the downvoters:

https://news.ycombinator.com/item?id=46911901

https://news.ycombinator.com/item?id=47457963

Analemma_ an hour ago | parent [-]

You're being downvoted because you posted a non sequitur, not because people don't believe you. Vulnerabilities in the OS are not the same thing as apps using the provided APIs, even if they are predatory apps which suck.

an hour ago | parent [-]
[deleted]