| ▲ | tedsanders 4 hours ago |
| Meta: The potential proliferation of biological weapons is serious. Millions could die. It's easy to joke about before it happens, but try to imagine how this thread might look after the successful deployment of a biological weapon by a rogue state or non-state actor. I encourage you to take this topic seriously and contribute posts that add new information or perspectives to the discussion. (I myself think the odds of a bioweapon attack remain low and have not yet been seriously accelerated by LLMs, but this is absolutely something the world should pay attention to.) |
|
| ▲ | sajithdilshan 3 hours ago | parent | next [-] |
| I don't think knowing how to build a bio weapon means one could actually build a bio weapon. It's like saying knowing how to build a nuclear weapon can easily lead to building a nuclear weapon. Advanced machinery and safety precaution is needed to engineer a virus or a bacteria to be a bio weapon. It's not something you can do in your tool shed at the moment. Also one would need lethal viruses to even start with and that's not something you can order off of amazon. Further, even to build a chemical weapon, the compounds needed are strictly controlled almost in every country and I would assume any suspicious purchase or order would immediately raise a flag or alert in national security service in respective country. |
| |
| ▲ | pixl97 an hour ago | parent | next [-] | | >not something you can order off of amazon. I mean there are services in which you can order things from wet labs so it's not completely hypothetical. | |
| ▲ | palmotea 3 hours ago | parent | prev [-] | | > Advanced machinery and safety precaution is needed to engineer a virus or a bacteria to be a bio weapon. It's not something you can do in your tool shed at the moment. I think the claim in this report is that it was a state or semi-state actor. They were from "blocked regions" at a "military research institute." So dismissing the threat by solely modeling it as a disgruntled layman rando is not reasonable. But you totally have to factor in the fact that anything coming out of Anthropic or OpenAI is part of a propaganda campaign to serve their business interests. These threats need to be addressed in ways that cause Anthropic and OpenAI pain (which means damaging their business). Because, FFS, anyone consciously racing to build doomsday devices needs a good, hard slap. |
|
|
| ▲ | AustinDev 4 hours ago | parent | prev | next [-] |
| How probable do you think it would be that a layman following ChatGPT's bio weapon instructions would just blow themselves up or poison themselves? It's not like positive pressure environments and suits are very common let alone all the other gear you'd need to safely synthesize arbitrary organic molecules. If a rogue state-level actor is doing this then why wouldn't they just kidnap a scientist and hold their family at gunpoint until they got them to do it for them? and if that's all it takes then why hasn't it happened yet? |
| |
| ▲ | 14u2c 3 hours ago | parent | next [-] | | Why does it have to be a layman? The 2001 anthrax attacks are believe to have been carried out by an insider to the field (senior biodefense researcher). Such tools could potentially be accelerants for similar individuals? | | |
| ▲ | AustinDev 3 hours ago | parent [-] | | If the five eyes intelligence services aren't monitoring all biodefense researchers and bio chemists around the world all of the time they've probably failed in their jobs. |
| |
| ▲ | techjamie 3 hours ago | parent | prev [-] | | All of this also assumes it doesn't hallucinate heavily in the process and give you instructions that are in reality utter nonsense, or create something entirely different from what you were trying to do. If you've managed to get the equipment and resources to pull this off in the first place, someone with the biological knowledge probably is not the ceiling stopping you from the other part of the problem. | | |
| ▲ | AustinDev 3 hours ago | parent [-] | | Yeah, back when this stuff was pretty new in 2024 I found a jailbreak and sent some bio/chem weapon instructions it generated to an organic chem PhD friend. They told me not only were the procedures wrong but that there were several steps that almost certainly would have lead to injury or death. I assume it's gotten better by now but, no way or desire to test. |
|
|
|
| ▲ | tredre3 4 hours ago | parent | prev | next [-] |
| I don't know why you emphasize rogue and non-state actors. It seems equally likely to me that it would be done by America. America is the only country who deployed nukes (Japan) and chemical weapons (Vietnam) on massive scales. I can agree that access to bioengineering tooling is easier and cheaper than ever, and thus eventually it stands to reason that a nobody in his basement could engineer a lethal novel virus and lose control of it. |
| |
| ▲ | BobbyJo 4 hours ago | parent | next [-] | | 1) The US is not the only country to deploy chemical weapons. Are you forgetting all of WW1? It's the whole reason chemical weapons are a no-no now. 2) The US didn't use chemicals weapons in Vietnam. Agent Orange was used to kill off foliage, not as a weapon against people, and the side effects were unintentional and affected US troops as much as Vietnamese. Edit: I originally noted WW2, but I was thinking of WW1's widespread use of things like mustard gas, and the resulting Geneva agreements. | | |
| ▲ | dubcanada 4 hours ago | parent | next [-] | | 2 - is a little pedantic, they used agent orange to kill foliage to kill people... They didn't gas people with chlorine or mustard, but they mass destroyed crops and foliage, to kill people. And "affected US troops as much as Vietnamese" is just completely incorrect, US covered a nation in herbicide, went home, got cancer. Is completely different then having your soil destroyed for decades. | | |
| ▲ | BobbyJo 4 hours ago | parent [-] | | > is a little pedantic, they used agent orange to kill foliage to kill people... Is a wheel a weapon because a tank uses it? I wouldn't consider the difference pedantry. | | |
| ▲ | dubcanada 3 hours ago | parent [-] | | It does look like the general consensus is agent orange is not a chemical weapon, or at least was not used as one. |
|
| |
| ▲ | Scea91 4 hours ago | parent | prev | next [-] | | 2) Unintentional side-effect affecting millions of Vietnamese people? Hundreds od thousands of children born with birth defects. | | |
| ▲ | BobbyJo 4 hours ago | parent [-] | | Is your argument that the resulting birth defects were intentional? If not, I'm not sure where you're going with your response. | | |
| ▲ | Scea91 3 hours ago | parent [-] | | The civilian cost was intentionally not part of the equation. The toxic effects were known and the numbers show that it was not a few isolated accidents but a intentional disregard for the human cost. |
|
| |
| ▲ | nbhkvl 4 hours ago | parent | prev | next [-] | | Chemical weapons didn't work well in WW1. The reason we don't see any chemical weapons around is because they are not really effective. Germans accidentally gassed a bunch of their own guys, while failing to use chemical weapons to achieve anything in the war. | |
| ▲ | well_ackshually 4 hours ago | parent | prev | next [-] | | "no see they didn't use it to give people cancer, they merely wanted to make the land they lived in so uninhabitable and devoid of life that staying there would mean death either by starvation or bombing" War crime apologists are always funny | | |
| ▲ | BobbyJo 4 hours ago | parent [-] | | I'm not sure what you're responding to. If you are making the claim that Agent Orange was indeed a chemical weapon, I'd disagree, but I think either perspective has points. |
| |
| ▲ | fhejfnenjdcn 4 hours ago | parent | prev | next [-] | | WW2 is notable for not seeing widespread deployment of chemical weapons. Hitler had been subject to gas attacks as a foot soldier in WW1 and forbade the use of chemical weapons as inhumane. | | |
| ▲ | btown 4 hours ago | parent | next [-] | | Per https://en.wikipedia.org/wiki/History_of_chemical_warfare#Wo... this (a) overlooks the widespread use of chemical weapons in the Pacific theater, and (b) ascribes the non-use by Germany to humanitarian reasons, which is largely speculative. Actual quotes from officers discuss far more practical reasons: Germany feared chemical retaliation, and the horses relied upon for logistics could not be outfitted with sufficiently performant gas masks. | |
| ▲ | fintler 4 hours ago | parent | prev [-] | | > Hitler had been subject to gas attacks as a foot soldier in WW1 and forbade the use of chemical weapons as inhumane. He didn’t seem to have a problem using them against civilian targets. |
| |
| ▲ | FpUser 4 hours ago | parent | prev [-] | | [flagged] | | |
| ▲ | BobbyJo 4 hours ago | parent [-] | | I'm not arguing we were the good guys in Vietnam, just that calling Agent Orange a chemical weapon is a shaky claim. Not sure why you're putting so many other words in my mouth. |
|
| |
| ▲ | tedsanders 4 hours ago | parent | prev | next [-] | | I emphasize rogue and non-state actors because they're more likely to be accelerated/enabled by LLMs. Large states like the USA already have access to modern bioweapon technology (and have a longer track record of not deploying them). | |
| ▲ | advisedwang 4 hours ago | parent | prev | next [-] | | There aren't many situations where biological weapons are useful to nation states, especially ones with global economic and military footprint like the US, because it isn't selective enough about who it targets. | |
| ▲ | theptip 3 hours ago | parent | prev | next [-] | | State actors already have the resources to do this. The disruption is from actors where the normal deterrence ladders don’t apply. | |
| ▲ | 4 hours ago | parent | prev | next [-] | | [deleted] | |
| ▲ | aqme28 4 hours ago | parent | prev | next [-] | | We're not there quite yet, but I think within the decade at this pace it may be possible to create targeted bioweapons for purposes of targeted assassination or genocide. It's horrifying, but a world in which everyone has a superintelligence is a world where that will sometimes happen. | | | |
| ▲ | yesitcan 4 hours ago | parent | prev | next [-] | | Because we’re the “good guys.” | |
| ▲ | bdbdjxjdbdb 4 hours ago | parent | prev [-] | | [dead] |
|
|
| ▲ | alansaber 4 hours ago | parent | prev | next [-] |
| Just like a dirty nuke you need a non-trivial amount of resources to fabricate a bioweapon, which would surely be noticed. |
| |
| ▲ | chucksta 4 hours ago | parent | next [-] | | They just found a hidden biolab in US earlier in the year which was connected to one found in 2023. | | |
| ▲ | groby_b 3 hours ago | parent [-] | | The details on the lab are (understandably and simultaneously frustratingly) quite sparse. But it's worth keeping in mind that theoretically, the mold on that cucumber in your fridge constitutes some sort of biolab. The main released criteria about those lab that raised alarm was the presence of specimens, which does not imply any capability of creating a weaponized strain. It's still alarming, but (at least in my opinion), still doesn't prove an effective bioweapon is buildable in a small-scale operation. Georgetown has a good article on the general problems with our current "AI and bioweapons" dialog: https://gjia.georgetown.edu/science-technology/rethinking-th... Without a whole lot of tacit knowledge no LLM can provide you, bioweapons are still pretty much out of range for most of the population. Doesn't mean we should ignore the problem, but a more reasoned approach would stand to benefit everybody. |
| |
| ▲ | iAMkenough 4 hours ago | parent | prev [-] | | Don’t worry, the Pentagon assigned a 19-year-old Jan. 6 rioter to lead counterterrorism efforts. https://www.washingtonpost.com/national-security/2026/06/02/... |
|
|
| ▲ | ishouldnotbutk8 4 hours ago | parent | prev | next [-] |
| >successful deployment of a biological weapon by a rogue state actor I was under impression that any medium or large state actor would have no problem developing biological weapons? They certainly have enough resources and talent. A much bigger problem is if every wannabe-terrorist suddenly could build a biological weapon in their garage. |
|
| ▲ | 4 hours ago | parent | prev | next [-] |
| [deleted] |
|
| ▲ | 0xWTF 4 hours ago | parent | prev | next [-] |
| The difference between nukes and bioweapons is energy level. Body counts are on the same order of magnitude. Bioweapons are just way easier to make and much harder to detect. |
| |
| ▲ | goatlover 4 hours ago | parent [-] | | Nukes also destroy infrastructure and irradiate surrounding areas making cleanup and rescue harder. Then there's to potential for massive fires putting debris into the stratosphere, which could dim sunlight for a time and cause global temperatures to fall. | | |
| ▲ | kian 3 hours ago | parent [-] | | bioweapons tend to not remain contained to the area they're deployed (with notable exceptions for things like Anthrax). Some of them even last a long time on surfaces, making cleanup and rescue harder. |
|
|
|
| ▲ | nbhkvl 4 hours ago | parent | prev | next [-] |
| Look! An OpenAI marketing guy talking nonsense again. I see no need to take anything you say seriously. |
| |
| ▲ | mdeeks 4 hours ago | parent | next [-] | | He doesn't seem to be a marking guy. Also you haven't given a reason not to take his words seriously | |
| ▲ | 123jas 4 hours ago | parent | prev [-] | | Too much pushback, so the AI people flagged this submission to the third page in minutes. Like Kabuki theater. |
|
|
| ▲ | 3412876 4 hours ago | parent | prev [-] |
| An OpenAI employee is trying to moderate the discussion? Do you remember that any Wuhan/GoF discussions prior to 2022/2023 were considered conspiracy theories? Maybe you should have intervened earlier! Instead, now when it serves the hype and the ClosedAI/Misanthropic valuations discussions are suddenly allowed. But you already write yourself that LLMs are useless for the task, like for any task apart from brute-forcing mathematician guided Lean proofs. Even if they weren't useless, you still would need a lab, which are already monitored and destroyed like in Iran. By people in the real world. |