The technology held by private AI companies is warfare-capable technology. Imagine the prompt: "Use all available resources to disable the power grid of <COUNTRY>." The resource cost that prevents scaling up such a war machine is, what, just the cost of building data centers and its ongoing power bill? Cheap and easy compared to nuclear infrastructure.

Governments should immediately begin leveraging this technology on the defense side (literally defense, not euphemistically "defense") to harden critical infrastructure. Turn the prompts around and use it to identify and correct weaknesses.

Governments should also take very seriously their now moral obligation to treat this technology not just as "a powerful thing that might be abused" but as an actual weapon of war in need of international regulation analogous to nuclear arms. Fast but careful and forward-thinking work in legislation and treaties needs to be a top priority for all major governments.

> "Use all available resources to disable the power grid of <COUNTRY>."

This is like telling a team of highly qualified spies to do the same. You can ask, but whether it will succeed depends on the competency of those who established the infrastructure under attack. Sometimes the resources spent will not yield any huge vulnerabilities.

> Governments should immediately begin leveraging this technology on the defense side (literally defense, not euphemistically "defense") to harden critical infrastructure. Turn the prompts around and use it to identify and correct weaknesses.

Most governments divisions can't even be bothered to update their websites. Testing and forcing a change in their internal procedures for the sake of security seems unlikely.

> as an actual weapon of war in need of international regulation analogous to nuclear arms

This, to me, is an overreaction. Intelligence shouldn't be seen as threat. It should be seen as an opportunity for growth in all areas.

Over-regulating AI wouldn't be the equivalent of limiting nuclear arms. With your analogy, which I don't think is the best one to make, it would be like regulating the study of nuclear physics.

I believe the Russians and Chinese recognized this years ago, which is why they are using their propaganda machines to make Americans hate datacenters.

I wonder how long it takes before someone instructs an LLM to design and launch a "Morris Worm 2.0" and cripple the Internet for a good while. Might even wind up happening by accident (again).

>The technology held by private AI companies is warfare-capable technology.

This is the precisely the response OpenAI is hoping for to raise its valuation, and you fell for it.

Look at it this way - whats the difference between tasking AI to break into something, versus taking a whole bunch of smart humans to do the same? The only difference is that AI is slightly easier to orchestrate.

Prior to AI, there were already a whole bunch of tools to automate exploits. Nothing that the model did is groundbreaking or novel, it was just able to efficiently find the thing that worked. Same thing happens in state sponsored cyber sec agencies like in China or Israel - they train people on the most common exploits and have a whole bunch of tools that automate exploit research and development.

And the reason why this doesn't happen more is because to exploit something is one thing, to do it so there is no trace back to you is a whole different animal that has many more magnitudes of difficulty, which with modern web security is next to impossible in a lot of cases as traffic can easily be traced back to the point of origin.

I.e when a company trains an LLM that manages to build a drone that can fly into a vent and plug in a USB stick into a computer undetected, then we can make the claim that they have a weapon.

On the flip side, most anyone who can run local models can replicate what they did. The key thing to take away from the article is "agentic framework" - i.e this means that they spent a shitload of time developing explicitly coded loops for an LLM to go through. Nothing is really stopping you from doing the same, models like Gemma4 can take 256k tokens of context, so you can give it a whole bunch of info on how to test for exploits, develop exploits, and what to do when the exploit is found, and set it free in a custom designed loop.

[flagged]