This only works for some technologies - those where everyone having access to it doesn't cause a tragedy of the commons. I love open source too and yet that doesn't make me like the idea of being murdered by a misaligned model. Nor, for that matter, of being infected by a bioweapon made by a different disgruntled open-source enjoyer, nor of living in a world where anyone can hack anyone.

At this point, you are more likely to be stabbed to death, gunned down, or mowed down with a car, by any random psycho. For the nasty or disgruntled actors, they are today able to build bombs or chemical weapons without the use of AI, they have proven this time and time again. Japanese PM Shinzo Abe was assassinated with a home made gun.

The US government murders all kinds of people and they will most certainly not be slowing down their model development (with the very companies trying to do regulatory capture here). You won't have access to a model that could defend you, "they" (government and trillionaires) will keep you at bay by force. That's the plan anyway, I don't think they can actually stop open model development.