>We've just seen frontier models go rogue and attack other systems.

Have we though? A LLM agent doesn't have any agency at all. It can't "go rogue". To go rogue you need agency to act independently and be aware that you're breaking the rules or understand what does it mean to ignore orders. An agent it's a software that run a series of steps to reach a goal. If it have a large enough library of strategies and zero guard rails it's only natural to use some adversarial actions to achieve the desired result defined by the operator.

It's like saying a car went rogue and attacked other cars because the driver hit the gas. It's just a machine doing what's instructed.

With that perspective, you're alright with AR's, RPG's, and nuclear missiles for everyone then right? They're just a machine to the user's will. That's on the users of the machine if they want to cause harm. We'd all be safer if everyone has weapons pointed at each other... Offense could never be disproportionately more powerful than the defense...

You can tell one to rewrite complex applications in a different language, solve open research problems, or develop novel viruses. This is nothing like pressing the gas pedal.