Just because the author can't think of a way to cause mass death with an engineered virus, doesn't mean an AI won't be able to think of one.

(There are several biological threats the author hasn't considered. I'll just leave it at that.)

▲

alevskaya 11 hours ago | parent | next [-]

I was a biophysicist and experimental biologist for decades, I designed viral vectors and cell therapies, etc. What protects us here isn't our ignorance vs an "ASI".

It's the fact that the complexity of molecular physics scales exponentially in particle number. No amount of thinking is going to punch through that. You've got to do experiments. The slow loop through reality is not optional, and in the case of "superviruses" would require a lot of iterative experimental development in humans.

(And please knock off the bio infohazard act - I routinely get asked about scenarios that AIxBio safety people cook up and they always involve some howler misunderstanding of basic facts in biology or medicine. Just talk about an idea if you have it.)

▲

andai 10 hours ago | parent | next [-]

>(And please knock off the bio infohazard act - I routinely get asked about scenarios that AIxBio safety people cook up and they always involve some howler misunderstanding of basic facts in biology or medicine. Just talk about an idea if you have it.)

Yeah, nice try ;)

▲

areoform 9 hours ago | parent | prev | next [-]

Hey there, is there some way for me to get in touch with you? I'm writing a piece on the topic, "Have you ever tried making a bioweapon?"

▲

naasking 10 hours ago | parent | prev [-]

We're already doing those experiments. Tailor made mRNA vaccines targeted to one's own specific cancer mutations can be bought right now. The techniques are getting more sophisticated and more targeted every year. Our ability to predict what happens at these levels is improving by leaps and bounds too thanks to AI like AlphaFold. Anyone can be reasonably confident that no AI or teen can do this now or in the next few years, but are you really so confident what might be possible in 10 years?

	▲	alevskaya 10 hours ago \| parent \| next [-]
		I worked in immuno-oncology: cancer vaccines work in melanoma where many things work because of the neoantigen abundance, they've generally been very mixed in efficacy. And even BionTech's BNT111 failed in melanoma! We have hope for these approaches but the reality of this stuff is way more nuanced than you think it is. Alphafold can't reliably predict thermal energy landscapes or make functional predictions - and how could it? It wasn't trained on anything that could capture structure - function relationships. Again, most people just have no idea how hard - fundamentally hard - molecular physics is to predict, and how necessary experiments are for any development of biological systems.
	▲	owebmaster 9 hours ago \| parent \| prev [-]
		> but are you really so confident what might be possible in 10 years? Do you think we are finally 10 years away from curing baldness?

▲

pama 10 hours ago | parent | prev | next [-]

I am not sure what you mean here. There exist plenty high-concern biological threats that dont need any AI help. Human oncovirus design is low on my concern list (immunity is diverse), and in any case it does not need AI—rather labspace. Disgruntled high schoolers or undergrad chemists can do way more damage from readily available materials without AI and without research delay. As can nature (or amateur biologists without AI) by mixing bats with their animal of choice and waiting a while. If/when any scary global events like covid happen again, I sure hope we have true superintelligence to help us navigate it quickly.

▲

phoghed 11 hours ago | parent | prev | next [-]

I know what you’re thinking. A modified HIV virus that also makes everyone hot and horny.

	▲	skeledrew 6 hours ago \| parent [-]
		A surge in rape cases and just curiosity into this new heightened-sex fad will lead to discovery pretty quickly.

▲

jeremyjh 11 hours ago | parent | prev [-]

The author can't think of what a superintelligence will do, because they are not a superintelligence. But that is not really what people are concerned about, so I think the objections here are quite meaningful.

If there is a fully general superintelligence - in the current state of alignment - we will all die for some reason. It doesn't really matter what the reason is.

▲

andai 10 hours ago | parent | next [-]

Well, at that point it doesn't even need to be a virus. It can just be nanobots that atomize your liver. The substrate is a mere technicality.

	▲	jeremyjh 10 hours ago \| parent [-]
		Right. A superintelligence will kill us in one or more of thousands of possible ways for thousands of possible reasons, perhaps none of which we may understand. Short of a superintelligence - the difficulties of viral bioweapon engineering the author points out are quite meaningful.

▲

cobbzilla 11 hours ago | parent | prev | next [-]

please define fully general superintelligence & current state of alignment

is this a tautology?

	▲	andai 10 hours ago \| parent [-]
		Not the guy but, alignment seems to be several decades away, while self-improvement seems to be several years away. (Also you don't even need superintelligence to do a lot of damage. In fact you don't need any intelligence!)

▲

11 hours ago | parent | prev [-]

[deleted]