Explain a path from today to the extermination of humanity in 2028 by the ASI and von Neumann replicators heading to the stars in the early 2030s without resorting to magical thinking or underpants gnomes level business plans. Engineering details are what matters here, not wishy washy things you once saw on Star Trek or The Outer Limits. Go.

I don't think we'd be okay with that if it took a few years longer, right?

I thought the Bengio blog post posted here the other day answered that question quite well (ignoring your timeline).

The argument is that in training for the capability to achieve certain goals, you may inadvertently also train for secondary instrumental goals you did not intend, such as aggressive behavior or deception.

I'm actually no sure I buy that this is particularly likely, or necessary, but I found that take reasonable and worth considering.

I think either "AI 2027" or "If Anyone Builds It, Everyone Dies" meet your criteria?

I guess not literally, due to the dates? But they have substantive technical details.

> Explain a path from today to the extermination of humanity in 2028 by the ASI and von Neumann replicators heading to the stars in the early 2030s without resorting to magical thinking or underpants gnomes level business plans. Engineering details are what matters here, not wishy washy things you once saw on Star Trek or The Outer Limits. Go.

Why? Those aren't the things I'm actually worried about, and those aren't the only "doomer" ideas, just the most extreme ones (thus easy to straw-man).

My main worries are about economic disruption, social damage due to the pace of technological change outpacing human ability to manage it, further concentration of power into small elites (along existing lines), and the reallocation of labor towards less personally-fulfilling pursuits.

I also totally buy into the idea that "AI leaders" are playing up certain risks as a marketing strategy. After all these are smart tech people, and those types have a tendency to get really arrogant and be irresponsible.

So the main thing is to slow all this down, so there's a chance of getting it under control, and snatch the payday away from such greedy and irresponsible people.

Have you ever lived through a power outage longer than 48 hours?

Did you see what it did to people in your neighborhood?

That’s the path.

Thats the problem with doom.

What might not affect you, might affect millions across the ocean.

A society no longer creating bullshit jobs because there is no reason for it anymore. You have some AI.

Right now a whole industry is transforming (again) but now filling the automation gap we were unable to do because of a shortage of labor or expertise.

Our society already has peak employmet reached, now AI takes away all the random digital jobs too. Someone building a website? Takes 5 minutes. Someone doing odd jobs on fiverr with blender or any other tool? Takes 5 minutes now.

Regression can happen very fast. The Romens were a high level culture and an empire. Gone.

Today you have more people on the planet then ever, everyone who might want to eat and who needs to work something for it.

AI already replaced people, already makes a lot of tasks A LOT cheaper and faster and human independent.

We never had a proper discussion as a society. Most countries have a social system but look at countries like the united states of america: if you are poor, you are fucked.

And AI is not the only thing which is now peaking: Physical AI / Robots are progressing fast, its clear today already that in 50 years these robots will be better than humans (50 years is very conservative).

If we have 10 more golden years due to restructuring and rebuilding things and preparing everything for agents and robots etc. what happens after these years?

If we have 10 or 20 more of these years, what do you think will happen to all the new kids (yours perhaps too?) and jobs and everything?

Riots? Famine? Mass migration? Mass death? Slums?

And don't respond with "enough people will survive this" bullshit. No one is saying that earth as a planet is not surviving. Its the same thing with climate change...

If someone showed you such a path, would you consider changing your mind? You're always going to be able to find a reason that such an explanation doesn't count if you're dedicated to looking for one.