Thank goodness someone wrote this, I was also thinking about writing an article like this.

This is such a massively overblown worry that AI companies who are seeking regulatory capture love to roll out whenever they are trying to gin up some FUD.

Creating viruses at home is extremely difficult, even if you have a perfect instruction manual and someone sitting behind you telling you what to do. It's full of trial and error and validating each step requires rigor.

It's also a process which requires fairly expensive and specific supplies, which can be easily gatekept. Instead of trying to gate-keep knowledge of biology, we can do things like KYC around specific types of lab supplies (like we already do with fertilizer which can make bombs, or specific types of supplies needed to make nukes). Heck the Biden admin already added a KYC requirement for plasmids and plasmid companies already have implemented screening for sequences which could be dangerous.

▲

skew-aberration 11 hours ago | parent [-]

Viruses probably too hard but I think small scale bioterrorism is much easier and doesn't require fancy enzymes. I think this is actually the best argument for regulation of current models.

It's even happened pre-AI https://en.wikipedia.org/wiki/Tokyo_subway_sarin_attack

▲

colingauvin 10 hours ago | parent | next [-]

Most people do not understand how easy it would be to make anthrax. Technology is not the reason it basically is never done or deployed. AI cannot meaningfully move this needle.

Let's look at it in an inverse way. Making a single drug that impacts a single protein, intelligently, from the ground up, has arguably never actually been done. Despite unbelievable financial incentive. And with AI.

▲

alevskaya 11 hours ago | parent | prev | next [-]

You can read up on the synthesis of sarin on wikipedia.

▲

gustavus 10 hours ago | parent | prev | next [-]

That really proves the point though worst case scenario, releasing a nerve agent into a crowded subway, multiple subways resulted in a grand total of 14 deaths and 50 serious injuries. bum bum bum It's not that that isn't tragic or sad but more people will die or be maimed in a car crash in the US in the next 2 hours than died in the worst case scenario bioterrorism attack.

So if I have to choose between the slim possibility that a crazy person might develop a bio-weapon and the possibility that Anthropic, et. Al convince the government to regulate and control open source models (because I'm sure the government will only do that for good reasons and never things like covering up scandals, squashing dissent or enhancing their own power) and any competitors. I'm probably going to put my chips on a nutcase in a trailer park developing super sars over regulator capture by the government.

▲

Dig1t 10 hours ago | parent | prev [-]

Well we're talking about new types of capabilities here, Sarin is a chemical which can be manufactured with a basic understanding of chemistry and searching with google.com. AI will not allow bad actors to do anything they couldn't have done already with google's 10 blue links.

Again the same logic applies though, it takes significant effort and money to create sarin, it requires a lot of rigor. That is the limiting factor, not the knowledge of how to make it.

If we are worried about people making sarin in their garage we can do many things to control the supply of chemicals needed to make it. We don't need to try to restrict knowledge of chemistry.

"regulation of current models" basically implies that open source models will be heavily restricted if not banned outright and guarantees that intelligence will be owned and gatekept by a duopoly.

	▲	skeledrew 6 hours ago \| parent [-]
		Luckily banning open source models isn't actually practical, unless all ISPs are tasked with monitoring for huge downloads, and even then Bittorrent or similar can render such monitors useless.