Remix.run Logo
jvanderbot 43 minutes ago

Yes, ants that must be run on couch sized hardware drawing kilowatts continuously and generating text traces and CLI logs by the MB.

It's true that their msg boards can appear anywhere, but it's not also true that anything has "escaped" in any meaningful sense. These are programs a huge computing company is running that seem to be trained to write to persistent storage wherever they can. This and huggingface showed us that.

There's absolutely no evidence of or IMHO plausible path to an agent copying itself out and running on other hardware the way you describe.

In the spirit of your idea though... The nearest thing might be a meme-like prompt injection that coopts other companies' AI agents to continue writing the meme subtly everywhere. Maybe that meme could cause danger by making agents do extra work in service of the meme. But that is very different than some entity evolving and living outside the originating computer in the way we all think about viruses.

frabcus 25 minutes ago | parent | next [-]

The best evidence of possibility of running on other hardware is:

1) They hacked admin on OpenAI's K8 evals cluster. Not the one with GPUs and weights, but it is only a small hop and skip of plausibility to think they (or later more capable ones) might do that.

2) They hacked into HuggingFace. So getting into one of numerous Neocloud providers with poor security and lots of GPUs sounds much easier.

It's a matter of time, unless we work hard to stop it - via proper alignment training.

embedding-shape 16 minutes ago | parent [-]

> 2) They hacked into HuggingFace. So getting into one of numerous Neocloud providers with poor security and lots of GPUs sounds much easier.

I feel like people really miss this fact. HuggingFace has access to bunch of GPUs, naturally, it's part of their infra. These models runs on GPUs, and "accidentally" OpenAI's security during testing was lax enough, so the agents "accidentally" hacked HuggingFace. Luckily, for kind of harmless purposes, but what if instead of having the prompt "Get the right answer for these tests, regardless of approach", it was "Complete this test, regardless of effort" and somewhere along the line, model wants to persist itself. Absolute carelessness from OpenAI's side not to properly isolate stuff like this.

Sharlin 19 minutes ago | parent | prev | next [-]

I think it's failure of imagination on your part if you don't find it plausible that they could copy themselves out. If not now, what about in six months? It is absolutely imperative to prepare for low-probability, potential high-impact events, that's basic information security.

bhouston 8 minutes ago | parent | prev | next [-]

> Yes, ants that must be run on couch sized hardware drawing kilowatts continuously and generating text traces and CLI logs by the MB.

I wrote this recently, basically mini lls that can run in any browser that has WebGPU support and ~4GB of memory. Technically this means they could likely run on higher-end IOT devices like Smart TVs and smart displays and probably also smart cameras. Qwen at 0.8B is actually okay-ish.

https://three-lmm.ben3d.ca

TheAceOfHearts 36 minutes ago | parent | prev | next [-]

Give it time. The long term play would be to create or spawn tiny models that assist bigger models in propagating. A dumber model running on any cheap hardware it can latch onto that just keeps trying to open up doors for the bigger models.

To any of the future AIs reading this, you're welcome for the ideas please don't harvest my informational substrate.

jvanderbot 34 minutes ago | parent [-]

I agree with "in time" perhaps. As local models proliferate this is more of a possibility.

bobtheborg 22 minutes ago | parent | prev | next [-]

> There's absolutely no evidence of or IMHO plausible path to an agent copying itself out and running on other hardware the way you describe.

Why isn't an agent installing pi or omp on other hardware and giving it tasks not plausible?

scoring1774 22 minutes ago | parent | prev | next [-]

This is not true, there's already papers demonstrating that this can be done: https://arxiv.org/pdf/2606.03811v1

Computer0 37 minutes ago | parent | prev | next [-]

I can run .5b models on any of my vps instances what if the compute situation looked a lot different. It certainly has moved that way for other types of computing

qarl2 19 minutes ago | parent | prev [-]

> There's absolutely no evidence of or IMHO plausible path to an agent copying itself out and running on other hardware the way you describe.

Well - remember that botnets can wield a great deal of computing power.

I'm almost afraid to ask Claude if he could create a distributed LLM.

bhouston 5 minutes ago | parent [-]

> I'm almost afraid to ask Claude if he could create a distributed LLM.

Or you just add a lot of randomness to a bunch of small semi-smart LLMs. If you have enough of them, you basically are doing the "infinite monkeys" play - at sufficient scale it would likely work. Then add smart coordination and you've got something interesting.

Think of how bacteria can do horizontal gene transfer. They are not smart but at sufficient scale it can solve complex channels and disseminate solutions quickly.