Wasn't Bill Gates mostly just saying terrorists could use AI to build a bioweapon? That sounds legit to me.

zozbot234 2 hours ago | parent | next [-]

It's not "legit", it's pointless scaremongering about entirely speculative future dynamics. Current LLMs can't "build" anything novel in this broad space. What if future AIs make it a lot easier for researchers to defend against plausible bioweapons? That's also reasonably possible, and would be a reason to deploy bio-capable AIs more broadly (with meaningful safeguards of course. But these are comparatively cheap because worthwhile bio research is resource-intensive already). At the very least, it'd be nice to have an AI model that doesn't immediately refuse to answer questions about junior-high Biology class.

▲

jswelker 2 hours ago | parent [-]

So a terrorist can't use a local llm to help grow anthrax? That is not farfetched at all.

AI democratized the knowledge needed to build bioweapons. Like how with a 3d printer anyone can build a gun with no expertise.

	▲	zozbot234 2 hours ago \| parent [-]
		It turns out that LLMs, especially local LLMs, tend to hallucinate a lot when thinking about anything that's overtly fiddly or technical. This is even more the case when they're in a domain that isn't a natural part of their training data. If you have to "jailbreak" the model to get it to talk, you're so wildly out of the expected distribution that you'd be crazy to trust anything it says. It's basically making up stuff as it goes along. These are foundational issues with how the models are created, not something that a bad actor can just hack around. (The biggest real safety issue in this kind of space is actually that the model might actively goad some unsuspecting victim into doing something incredibly dumb and dangerous to themselves as much as possibly others. IIRC, there were reports of something vaguely similar happening IRL but involving casual mischief, not any kind of extreme attacks. And because nobody else seems to have managed to elicit the same actively goading verbiage from the model, it's implicitly suspected that the person involved was the one who introduced the problematic scenarios to begin with.)

▲

nradov 2 hours ago | parent | prev | next [-]

It only sounds legit to people who don't understand biology and haven't done advanced laboratory work. Sort of like Michael Crichton novels: superficially plausible but not grounded in any real science.

▲

jswelker 2 hours ago | parent [-]

I see no actual argument here.

▲

nradov 2 hours ago | parent [-]

You haven't made any argument either, so I suppose we're even.

▲

jswelker an hour ago | parent [-]

My argument was that it is reasonable to be concerned that human terrorists might use AI to assist with building bioweapons.

▲

nradov an hour ago | parent [-]

That's merely an assertion unsupported by any reliable evidence, not an actual argument. In short you're just making shit up.

	▲	jswelker 35 minutes ago \| parent [-]
		Well that's the nature of trying to prevent a thing that hasn't happened yet right? What would be reliable evidence except that terrorists already used AI to help build a bioweapon? I'm sure before 9/11 talking about terrorists using commercial airplanes as makeshift missiles seemed like scaremongering too. I see zero technical reasons why it could not be done, so being concerned about prevention seems pretty reasonable. I'm not saying AI uses robotic arms to build a bioweapon unassisted or something, just that it dramatically empowers bad actors enough to make them capable of things they previously were not.

▲

Barrin92 2 hours ago | parent | prev [-]

terrorists can also use annas archive, scihub and equipment they order on Alibaba to build bioweapons and I don't see him losing his mind over those

if you're trying to build a bioweapon shockingly enough the bottleneck is... the laboratory work. What on earth is 'legit' about stringing words together that sound scary, you can't just iterate 'ai bioweapon cyber' in a sentence over and over as if that adds up to actual evidence for an increased risk of any threat. well tbf you can technically because apparently it freaks a lot of podcast listeners out

	▲	jswelker 25 minutes ago \| parent [-]
		As a thought experiment imagine you have a biochemical PhD expert on the phone with you giving you step by step instructions on how to grow some dangerous biotoxin, giving you feedback in real time and helping you troubleshoot. Would you be more successful with this expert than you'd be on your own? Now imagine anyone can call that expert for free at any time. Maybe the AI isn't quite there with biology knowledge yet (doubtful), but it is a matter of time. How is that not a real risk?