Wow, this comment thread clearly shows that at least Anthropic has not been a great communicator.

If one reads this with a charitable lens, Dario is simply saying that 1) Nation state actors are a threat which needs to be combatted by chip bans and distillation prevention and 2) open-weight models can pose biological risk.

One may or may not agree with item 1 but item 2 above should have broad support given the unknown unknowns in play?

This argument is old and goes back to cyber security. Things like government backdoors are the same. It's true centralizing power in the hands of a "good" would theoretically be good. But the key part is the "good". I think after repeated experiences with governments and corporations, no rational person would ever assume them to be good. There are simply too many incentives to be bad.

Closed-weight models can also pose biological risk, arguably more so than open-weight models, given the fact that they're being used by state actors (the United States military) to kill people right now.

Who should we fear more? All of collective humanity with the keys to build destructive (and defensive) stuff with AI, or small groups of elites, billionaires, and state actors who have the monopoly on violence and want to control the keys?

Open-weight models collectivize access and ability to do more for a greater good, and the expense of a frankly low-risk possibility that some randos want to use it for very bad things.

Closed-weight models keep the control in the hands of the few that actually are doing the harm to the world, and the rest of us have no way to stop it or defend.