Can somebody tell me a story how this will unfold? And - as long as the AI is confined to data centers - how it will prevent humans from unplugging the power?

I don't think extinction-level events or something like killing a majority of the human population is particularly plausible at this moment. States don't host their nukes with AWS and a permanent connection.

But you could create scenarios where an AI with very, very roughly the current capabilities could potentially nuke everyone. Let's assume an agent decides that the way to solve its task was to get the US to fire all nukes on Russia. The agent would need to hack some government systems to understand how exactly to access them. Then it would need to get the content of the card with launch codes the president has, and identify which code is the correct one. Maybe that information is available somewhere and it can get to it, I obviously can't know that.

Then it could fake a call from the president, synthesizing his voice and ordering a nuclear strike. If it hacked enough systems to get into whatever communication pathways would be used in such a case. Would the soldiers listen to this order and follow it, I don't know.

Or maybe the agent can get in somewhere in between, to avoid the need to know the president's launch code. And fake a call from a military commander to the launch sites.

I think other scenarios that would cause significant harm, but aren't as bad as nuclear war are more plausible. And in those shutting down all data centers would probably be the way to stop it. The AI can probably hide in other datacenters, once it is at a point where it's running amok with some bad goals. But if it presents a huge and immediate threat at that point, humans will also go to great lengths to stop it.

I heard the following analogy which made a lot of sense to me: suppose you're playing a chess match against Stockfish. Stockfish will win. Even if I cannot tell you what moves it will play, I can tell you with certainty how it will end.

Similarly, we cannot predict what AI would do.

Very basic idea: A model breaks out by accident, finds some computer system from a military system and triggers some weapon system. Before anyone understands that this happend -> WW4 (WW3 is for me already the Conflict with Russia / aka proxy war).

Another model: Because we give AI Agents already that much power, imagine in 10 years everything running through an Agentic AI Layer. EVERYTHING. Now some rough system 'thinks' about something, starts to push through the then existing agentic ai layer systems and stops everything. Billions of humans would loose access to food and water, even if this is just for a short period.

Covid showed how shitty a handful of people can disrupt global supply chains. Toilet paper was. ahuge stupid pseudo issue in germany.

It is true that, currently, AI does not have any physical "host" in which to reside.

However, the missing link in this reasoning is that AI will almost certainly become far more widely deployed in the future than it is today, and worryingly, consumer devices will surely become powerful enough to run capable AI systems locally.

Once the substrate will be there, once an AI escapes containment, we're toast - I can imagine only solution will be to shutdown electronics at global level.

How will you know when to unplug the power? How will we know it hasn't replicated? A true unaligned AGI is a APT. If you have an APT in your machine, you have to rip out everything. Are we going to do that with all of our computer infra?

This is all still "what if's", but the tail end's are truly F'd beyond our ability to fix.

Maybe by empowering the small percentage of sadistic humans who want to kill everyone. Or maybe it is more of an academic assumption that humanity will end at some point, and so they give AI a 10% chance, asteroids have a 25% chance, nuclear fallout has 15% chance, etc.

Not exactly scientific but it is at least entertaining and some things do sound plausible https://www.youtube.com/watch?v=Gw_hnD7m00M

I believe the contention is we'll have some form factor of AI on edge devices, in reactors, in critical infra and weapons etc etc

For starters, how bad do you think it would be to unplug all datacenters? How many people starve?

Autonomous drones + false flag attacks

edit: wtf, why did I just receive so much gift tokens on my openai account?

You can run a model on your laptop, it is already not confined to data centers.

"Hello, ex-military person. If I put $10,000,000 worth of bitcoin in your wallet, can you do X for me please?"

What makes you think they will be confined to data centers?

What's more, AI just needs to have a credit card and it can start commissioning humans to do things for it in the real world.

Robots