I like it. If the increase in unrestricted cyber capabilities can be slowed down, please be it. The world needs to get ready, and that takes time. Way more time than even a slow progress would allow.

Unless you want to see more of what happened the past few days.

I think the argument that "cyber capabilities should be democratized, thus keep producing them now, fast, and for everybody" is a very naive one.

▲

LinXitoW 30 minutes ago | parent | next [-]

How does "the world" get ready, when these limits are so hard core that Opus regularly stops working when I'm just trying to find and fix security exploits in my apps?

Currently available, unhindered models can already find a bunch of exploits, not because they're smart, but because they're cheap and smart enough to just keep going for hours or days until it finds something.

The defenders, especially in the FOSS space, only have one big advantage: Unified scale. If defender A finds a vuln, then defender B doesn't need to find it again. Attackers generally act alone, so they need to do all the work themselves.

If we remove a bunch of defenders, the attackers are back at it again.

▲

2001zhaozhao 3 hours ago | parent | prev | next [-]

I am glad that at least one Chinese company seems to be listening to AI cybersecurity concerns and did concrete steps that actually reduce the problem. This was probably at least somewhat costly to them in terms of general model capabilities as well. They'd probably have gotten slightly better coding capabilities if not for removing the cyber training data and tasks.

	▲	AspireOne an hour ago \| parent [-]
		I am glad about it as well. Don't get me wrong, I love OSS, I love personally being able to get my hands on the frontier stuff and frontier cybersec capabilities, and I dislike a few entities having exclusivity on it. Everybody does. But that absolutely doesn't mean it's the correct approach. I think people are blinded a little bit by general principles and by personal feelings, rather than operating from a point of pragmatism, consequences, and the big picture. Also, to note: I don't claim to have the answers either. I don't even hold a strong position on this issue yet. I'm just sharing some of my intuitions, and reacting to people who, as far as I'm concerned, are making this issue all too simple.

▲

TuxSH 2 hours ago | parent | prev | next [-]

Most low- and medium-handing fruit vulns can be found and exploited by GLM and some knowledge already.

Kneecapping cyber vulns only serve governments, Pandora's Box is already open for cyber-related stuff.

	▲	AspireOne an hour ago \| parent [-]
		The Pandora box can expand even further, and it matters. If it didn't matter, like you seem to be suggesting, then by the same logic there isn't a reason to push for further open-source development. > Kneecapping cyber vulns only serve governments. Uhh, no? Kneecapping cyber vulns server most of us. There are negatives to kneecapping it, but let's not pretend like it's perfectly okay and safe for everybody in the world to have a magic box that can hack anything and cause havoc all over the world. The more powerful, cheap, and accessible this magic box is, the worse the consequences. This is a hard problem and there's real tension - like you said, the playing field not being leveled is a problem (major companies / entities / govts having exclusivity on the best cyber abilities etc), but everybody having access to it is a problem as well. So let's not pretend like this is a simple problem, and don't say that it only serves the govt. That's very silly.

▲

jrflo 3 hours ago | parent | prev | next [-]

Yes, thank you Moonshot for not throwing open Pandora's box and seeing what happens security wise.

▲

0xbadcafebee 3 hours ago | parent | prev [-]

> The world needs to get ready

How will it do that without a model that's good at security? You gonna have humans manually secure the millions of lines of new AI-generated code, in addition to the existing code? Suddenly people will start doing the job they haven't been doing up to now? Not only are we not capable of auditing all the code, it would take decades to finish, and new code will be coming out at the same time. We can't catch up by hand.

The only way to fight this fire is with fire. We need AI hacking tools to find our security holes quickly and patch them. Otherwise we're just churning out more insecure software, and the only people who can defend against it are the governments who don't want to patch the holes (because they want to be able to exploit them). Without open tools, we all lose.

	▲	AspireOne an hour ago \| parent [-]
		I disagree with this completely.