a danger could be the OpenAI/Antropic servers are up but there's a rouge agent (or set of agents) out there doing naughty things leveraging the LLM APIs. Consider this scenario, the agent is copying itself around (some code, prompts, persistent storage for memory, etc) and has figured out a way to steal API access tokens at will. Currently, it's 10% of OpenAI and Anthropic API usage and they can't figure out how to stop it.

Do you shut down the entire API and kill the legit 90% of usage to stop the rogue 10%? I'm assuming the providers would say "no way jose" and so it would take law enforcement to do it. That would mean all the legal requirements neccassary to walk into a business and flip the switch which i think would get tricky when there's no human committing a crime or being suspected of a crime.

edit: I guess a trivial example is something i did yesterday. I have a stock trading agent running on my laptop, i gave it ssh access to a vm and said "start running on the server so i don't have to keep my laptop open". It's now running on the server instead of my laptop. So you don't have to copy the whole model around to copy the naughty behavior around.

This assumes all layers of cybersecurity are broken - We call self-replicating software a virus, and we have protections against it. Same with stolen API tokens, just rotate them. Suspicious behaviour, nothing new, we have detectors for it. Stolen CPU / GPU cycles, we had that when crypto was a thing and before that when folding@home was cool, people were desperate to find more compute to the point of taking over systems. And we dealt with it.

A lot of the supposed risks / dangers are based on a supposition that cybersecurity is nonexistent or fatally, unfixably flawed and that AI agents are invisible. Neither of those is true.