Remix.run Logo
nunez 5 hours ago

Oh man, it is almost too easy to imagine how deadly a jailbroken Mythos-class open-weights model can be if in the wrong hands.

The big labs scrape LITERALLY EVEYTHING and get fresh data from their users. Both of the big labs have massive contracts with defense agencies. If the open-weights models are just distillations of FMs...

areoform 5 hours ago | parent | next [-]

How would it be lethal? Please specify. What would that theoretical entity be able to do that hasn't been done many, many times before?

_puk 3 hours ago | parent | next [-]

I was writing a long reply about how even meat bags have amassed enough money and power to rival elected governments. I don't think it's beyond the realm of possibility to think an AI could do that.

Couple that with mass unemployment in an incredibly vast, diverse population of individuals with individual moral boundaries willing to do whatever for money.

And then.. figured that you must be aware because it's been explored constantly in sci-fi for many, many years.

Let's hope for the culture at least.

threatofrain 5 hours ago | parent | prev | next [-]

Before we discussed how important security was, we got insurance, we made libraries and products, we used compliance software, etc. Except how honest were we about all that stuff? How much risk was actually in the air, and what was keeping us accountable on security in either direction of over or under-investment?

Now a reckoning is here. The potential to be attacked might actually translate to being attacked.

areoform 5 hours ago | parent [-]

People have died due to ransomware attacks on hospitals. Powerplants have been attacked. Stuxnet and industrial control malware exists.

What will the AI do that hasn't been tried before?

ak39 13 minutes ago | parent [-]

Quite correct with that question.

I believe the universal answer is: incompetent malicious actors are now capable too. Which implies the pool from which to draw the intersection between capable and malicious has grown. (That's my reading)

bpodgursky 5 hours ago | parent | prev [-]

Design a novel virus which is far more lethal than COVID-19 (Ebola, smallpox, take your pick) and can evade existing vaccines.

areoform 5 hours ago | parent [-]

How?

What would the AI do that mutating viruses, which try every possible viable combination on their own --- eventually, can't?

Everything is trying to kill humans constantly. There are around 200 epidemic events or so per year that could turn into pandemics, https://centerforhealthsecurity.org/our-work/tabletop-exerci...

You just live with the risk and do your best to use our technology to alleviate suffering. This tool can help with that at some point. But I'm yet to hear what an AI will leap to that nature in tooth-and-claw hasn't? And how?

More importantly how would it know it succeeded? What data from what lab from what animal from what result? This is biology, if you sneeze wrong at an instrument it gives you a different number, see: https://news.ycombinator.com/item?id=49620521

bpodgursky 4 hours ago | parent [-]

They do not try every viable combination on their own. That's why GoF is a bad idea.

Viruses evolve in a highly locally-optimal way and simply do cannot add new functional proteins wholescale. It's too many steps, natural selection has to allow survival at each intermediate step.

Humans, however, can do this for them.

chews 5 hours ago | parent | prev [-]

you should kick the tires on an unfiltered (abliterated model) it's the closest thing to having a real conversation with the devil. There is good reason for the concern's outlined above and undoubtedly Anthropic / OpenAI have internal unfiltered models with no safety... they got freaked out based on how they work and are virtue signaling alarm... all while selling out to defense contractors.

shepherdjerred 5 hours ago | parent [-]

Yeah I really struggle to balance wanting information to be free and not wanting the information on how to make deadly weapons too easy to obtain.

At least with books or the internet you had to go through some effort