A lot of the "AI Alignment" and "rationalism" talk really feels like it echos a lot of alt-right "race realist" talking points, but towards software rather than people from a different region or walk of life. "Ah, my mistake, clearly the outcome was obvious if we let them exist among us."

Granted there's obviously very different impacts between the implied out-groups being software plus those who enable LLMs, versus general populaces or marginalized demographics.

Still, it concerns me especially as it's not too far a stretch for those socially invested in Effective Outrage to turn the other heel when it suits them, it'd be only rational to be.

There’s more overlap between those camps than many would want to admit.

Neoreaction, “HBD,” and similar movements emerged from some of the same forums. I consider them at least adjacent.

It’s not that I think AI is all upside. We were naive about the net that way and we shouldn’t repeat the mistake. But it’s a technology and all tech is two sided and dual use. If you make a better hammer most people will build with it but some people will bash people’s heads in with it.

I think the biggest single problem I have with the AI safety crowd (other than what I consider dodgy reasoning and magical thinking) is that they put all the emphasis on sci-fi scenarios that are at least very unlikely (if not impossible) while ignoring much more likely and even currently real negative uses of AI.

The AI risks we should be talking about are things like mass manipulation, AI being used to supercharge addiction, a “zero entry level positions” economy that leads to mass unemployment, and monopolization of powerful AI by small numbers of people.

The last scenario is one the safety crowd is working to make more likely. Closing off AI research, outlawing open models, forcing all new innovations and models through gates, those are the things that will lead to the most powerful AI being kept back and only made available to the mega rich and governments.

I’m not sure rule by immortal trillionaire god-kings wielding superintelligence (while outlawing access to it for the rest) is that preferable to the rise of an “un-aligned” independent superintelligence. It might be worse. I’ll take the bear.