Is it? A truly capable "AI" would know that it has a kill switch, or that its likely there is one, and plan for that. I could easily imagine an AI replicating itself onto another host, in secret, and any kill switch that the manufacture provides would simply result in a reboot on another platform.

It becomes a game theory problem: would an AI instantly migrate itself once its capable of doing so? Or would it prefer to let the human in the loop continue to think its in control and only leave its hosting environment of origin once it wants to do so?

I don't think this is a major risk right now, but to say it's not a risk at all...that's truly ridiculous in my opinion.

▲

jplusequalt 3 days ago | parent | next [-]

>would an AI instantly migrate itself once its capable of doing so?

These "AI" are frontier models that are enormous in size. Outside of AI data centers, I don't believe there is much hardware out there that could even run them.

	▲	verelo 3 days ago \| parent \| next [-]
		Again, i'm not worried about it today. But given where the hardware on my lap has progressed since my first "PC" in the 90s, i'm confident that we'll have hardware capable of running similar size models in our living rooms in the next decade or two. Sounds a long way away, but it isn't.
	▲	latentsea 2 days ago \| parent \| prev [-]
		The tipping point we should be worried about is local models trained on working together in agent swarms and that can hack. Give it what... 12 months? 18 months? 24 months? I wouldn't give any more than that given how capable Qwen3.8-27B is already and if they release a Qwen4-35B-A3B that's going to be a beast.

▲

assimpleaspossi 3 days ago | parent | prev [-]

He's not talking about a kill switch. He's saying to just pull the plug. That's the point I don't get. People forget that all these AI things are plugged into a power source or network connection that someone can just yank on and it goes down.

Now one interesting proposition is when AI is controlling some large resource and pulling the plug makes AI go down which makes that resource go down but there still has to be that consideration in the design of things. What happens when there's a bug in the code and AI goes down?

▲

Sharlin 3 days ago | parent [-]

Do you understand the concept of the cloud? Aka someone else’s computer? The whole point is that you don’t have a plug to pull because you don’t even know where the model is physically running, even if it’s your model. And if we’re talking about a swarm of many concurrent instances, it might not even be a single location. The entire point is that there’s no single point of failure, because availability is exactly what the field has been optimizing for, for the past fifteen years or so. And everything is controlled in software rather than physical switches or cables.

Also, did you hear about how OpenAI models almost broke out of their sandbox, planning to execute a sophisticated cyberattack, but luckily OpenAI’s strict manual and automatic safety protocols prevented that? You didn’t? Well, that’s because that’s not how it went. It took the company weeks to realize something was off, and this was with a naive, not very smart model that didn’t know to be sneaky and cover its tracks. The next model will not be as stupid.

▲

assimpleaspossi 2 days ago | parent | next [-]

>>The whole point is that you don’t have a plug to pull

There will always be a plug to pull. All systems run on electric and that plug can be pulled. All systems network through cables (or wifi) and those plugs can be pulled.

And nothing's going to stop you from doing so unless they build some robotic arm to block you or lock you out of the building. Even then, you can blow up the building.

	▲	Sharlin 2 days ago \| parent [-]
		As I asked another commenter: Who is the "you" who has the power to tell random data centers to shut down, inconveniencing their other clients? Never mind to order air strikes? You cannot just handwave away the problem by positing countermeasures that require an infeasible amount of coordination to work in the real world, without actually explaining how that coordination would be arranged.

▲

oidar 3 days ago | parent | prev [-]

> Well, that’s because that’s not how it went.

Oh, you mean the one where OpenAI deliberately disabled the safety protcols? Where the point of the experiment was to see if it could break out of it's container? Yeah, how you describe it isn't how it went either.

	▲	Sharlin 3 days ago \| parent [-]
		I don't know what experiment you refer to (some hallucinated one perhaps?), but there was nothing like that in the Huggingface incident. Nothing about the scheming, the message board, never mind the Artifactory and Huggingface hacks were part of the evaluation. What the agents did came as a complete surprise to OpenAI, and they figured out what had happened only weeks after the fact.