Something I've been worried about for years is the proliferation of technologies that give ever smaller numbers of people with smaller amounts resources the ability to impact billions of people, in potentially irreversible ways. Combined with automation, this is a recipe for disaster. We're clearly lacking in the ability to regulate and govern ourselves, and real world market forces (as opposed to the hypothetical free market) encourage reckless aggressiveness at the level of companies and countries.

Although, I must admit, as a doomer, I'm pleasantly surprised things aren't already worse than they are.

This is what AI safety activist Connor Leahy calls offense-defense asymmetry. Imagine if a nuke could be created with household materials. We got lucky that wasn't the case, but we didn't know that it wasn't the case before studying physics. If we weren't so lucky that physics happened to turn out that way, we already wouldn't exist, because the offense would be so much stronger than defense, we wouldn't stand a chance even with the best surveillance systems.

A profitable prediction markets trader who goes by the alias semiotic rivalry posted this prediction on his X account recently:

> Prediction: in 1-2 years, almost everyone will agree that open sourcing the best AIs is totally crazy as the AIs become capable of autonomous hacking of most humans and institutions, serious terrorism and creation of biological weapons, etc.

And I mostly agree with him. As much as it's awful to have such a concentration of capabilities in the hands of a few governments or corporations, as far as I can tell there are no easy answers here, only a matter of trade offs and different risks.

Im fairly certain that the knowledge necessary to build a nuclear bomb is far less well-protected than the general public is led to believe. The only part that is actually difficult is obtaining a sufficient quantity of enriched uranium without somebody dropping a JDAM on your lab.

If the Manhattan project could build a fat man in 4 years your state university's physics department has somebody who could reproduce that given the necessary resources. It might take them longer than four years given that they are not staffed with the modern equivalents of Oppenheimer, teller, et al but they could do it within a human lifetime if they actually wanted to cause a mass-casualty event and weren't afraid of the aforementioned JDAM.

> proliferation of technologies that give ever smaller numbers of people with smaller amounts resources the ability to impact billions of people

did you mean greater number of people with less resources?

EDIT: thanks to others that point out how you can interpret this differently, which I think is: "it is now easier for smaller teams with fewer resources" rather than my original interpretation of "power is now concentrated so that fewer people can do this than before"

I think you would get a kick out of reading this paper on this exact topic, called the Vulnerable World Hypothesis https://nickbostrom.com/papers/vulnerable.pdf

However I think the result is disappointing, it concludes the only solution is a massive preventative surveillance state and global governance.

I’d say that is almost the definition of technology: that which allows doing ever more with fewer and fewer people and resources.

And technology doesn’t really care what you use it for. It can be constructive or destructive.

It “only” enhanced local destructive potential, until last century. Then it got to the point where a huge effort could produce a truly global catastrophe. And then a bit smaller effort. And technology continues to march on.

Every person's utopia is someone else's dystopia and vice versa. What you've described is clearly some peoples (ie. already billionaires) version of the best possible world. Not so much for the rest of us though...