This is also my reasoning for why I think AI alignment is not going to be a problem for humanity any time soon.

By the time AI will be capable of maintaining the whole supply chain required to keep itself running sufficient time will have passed so we can come up with something viable.

Long before 2100, critical AI system will no longer be operating from this soil. They are in Earths orit, and on its moon.

> AI cannot run if society collapses.

That doesn’t mean some idiot billionaires huffing each others’ farts can’t think it can.

Respectfully disagree. An AI with full access to robots could do everything on its own that it would need to "survive" and grow. I argue that humans are actually in the way of that.

If you're an AI company and you believe your own hype (like Musk seems to), you'll probably believe that you can automate everything from digging minerals out of the ground all of the way up to making the semiconductors in the robots that dig the minerals.

As you may infer from my use of the word "hype", I do not think we are close to such generality at a high enough quality level to actually do this.

This is also why I'm skeptical of claims that it would be impossible (or nearly so) for governments to meaningfully regulate AI R&D/deployment (regardless of whether or not they should). The "you can't regulate math" arguments. Yeah, you can't regulate math, but using the math depends on some of the most complex technologies humanity has produced, with key components handled by only one or a few companies in only a handful of countries (US, China, Taiwan, South Korea, Netherlands, maybe Japan?). US-China cooperation could probably achieve any level of regulation they want up to and including "shut it all down now." Likely? Of course not. But also not impossible if the US and China both felt sufficiently threatened by AI.

The only thing that IMO would be really hard to regulate would be the distribution of open-weight models existing at the time regulations come into effect, although I imagine even that would be substantially curtailed by severe enough penalties for doing so.

This is the best argument I’ve heard against it, so thanks.

My anxiety entirely orbits around the scale of AI compute we’ve reached and the sentiment that there is drastic room for improvement, the rapidly advancing state of the art in robotics, and the massive potential for disruption of middle/lower class stake in society. Not to mention the general sentiment that the economy is more important than people’s well being in 99.9% of scenarios.

Who's to say it has to keep moving forward? The companies are buying up massive amounts of GPUs in this AI race, a move that's widely questioned because next year's GPUs might render the current ones outdated[0], so there will probably be plenty of GPUs to go around if the CEO demands it (prior to collapse). Operating datacenters would probably be out of the question with a collapsed society as the power grid might be unreliable, global networks might be down and securing many datacenters would probably be difficult, but there's at least one public record of a billionaire building his own underground bunker with off-grid power generation and enough room to have his own little datacenter inside[1]. "Ordinary" people will acquire 32GB GPUs or Mac Studios for local open-source LLM inference, so it seems likely billionaires would just do the next step up for their bunker and use their company's proprietary weights on decommissioned compute clusters.

[0] https://www.cnbc.com/2025/11/14/ai-gpu-depreciation-coreweav... [1] https://www.businessinsider.com/mark-zuckerberg-hawaii-under...

I guess it's whoever was in that Doug Rushkoff meeting with the whole idea we'll have security forces with those exploding dog collars to keep them in line and to keep revolutionary forces from killing us and taking our food supply!

https://english.elpais.com/technology/2023-09-20/writer-doug...

it's very easy to achieve great things without coordination if you can just do what's best for yourself and help your peers achieve their collective goals.

but they do meet at davos every now and again, without the democratic shackles.

I don’t know if I believe it’s an active conspiracy. Instead I think it’s more of a very concerning, very plausible eventuality.