Viruses probably too hard but I think small scale bioterrorism is much easier and doesn't require fancy enzymes. I think this is actually the best argument for regulation of current models.

It's even happened pre-AI https://en.wikipedia.org/wiki/Tokyo_subway_sarin_attack

Most people do not understand how easy it would be to make anthrax. Technology is not the reason it basically is never done or deployed. AI cannot meaningfully move this needle.

Let's look at it in an inverse way. Making a single drug that impacts a single protein, intelligently, from the ground up, has arguably never actually been done. Despite unbelievable financial incentive. And with AI.

You can read up on the synthesis of sarin on wikipedia.

That really proves the point though worst case scenario, releasing a nerve agent into a crowded subway, multiple subways resulted in a grand total of 14 deaths and 50 serious injuries. bum bum bum It's not that that isn't tragic or sad but more people will die or be maimed in a car crash in the US in the next 2 hours than died in the worst case scenario bioterrorism attack.

So if I have to choose between the slim possibility that a crazy person might develop a bio-weapon and the possibility that Anthropic, et. Al convince the government to regulate and control open source models (because I'm sure the government will only do that for good reasons and never things like covering up scandals, squashing dissent or enhancing their own power) and any competitors. I'm probably going to put my chips on a nutcase in a trailer park developing super sars over regulator capture by the government.

Well we're talking about new types of capabilities here, Sarin is a chemical which can be manufactured with a basic understanding of chemistry and searching with google.com. AI will not allow bad actors to do anything they couldn't have done already with google's 10 blue links.

Again the same logic applies though, it takes significant effort and money to create sarin, it requires a lot of rigor. That is the limiting factor, not the knowledge of how to make it.

If we are worried about people making sarin in their garage we can do many things to control the supply of chemicals needed to make it. We don't need to try to restrict knowledge of chemistry.

"regulation of current models" basically implies that open source models will be heavily restricted if not banned outright and guarantees that intelligence will be owned and gatekept by a duopoly.