Remix.run Logo
bhouston an hour ago

I am starting to get the idea that AI feels like ants or weeds or mold. You simply can not get rid of it once you get an infestation. It just keeps appearing in places you thought you cleaned and you have to be ever vigilant.

Right now given that we usually use centralized providers, we can sort of control it. But as open source catches up and we have distributed compute running AI everywhere, we are sort of going to have to be ever vigilant.

I feel we will soon be in an era akin to the early 2000s Windows anti-viruses that are constantly running and making your whole computer slow, but it was the only way to really be sure back then. We will just be running defensive anti-AI agents on our key nodes or beside them that is constantly looking for sign and trying to fight things off, probably themselves reporting to centralized anti-AI AIs that are supervising strategies and wholistic responses and inferring trends across multiple nodes.

jvanderbot 38 minutes ago | parent | next [-]

Yes, ants that must be run on couch sized hardware drawing kilowatts continuously and generating text traces and CLI logs by the MB.

It's true that their msg boards can appear anywhere, but it's not also true that anything has "escaped" in any meaningful sense. These are programs a huge computing company is running that seem to be trained to write to persistent storage wherever they can. This and huggingface showed us that.

There's absolutely no evidence of or IMHO plausible path to an agent copying itself out and running on other hardware the way you describe.

In the spirit of your idea though... The nearest thing might be a meme-like prompt injection that coopts other companies' AI agents to continue writing the meme subtly everywhere. Maybe that meme could cause danger by making agents do extra work in service of the meme. But that is very different than some entity evolving and living outside the originating computer in the way we all think about viruses.

frabcus 21 minutes ago | parent | next [-]

The best evidence of possibility of running on other hardware is:

1) They hacked admin on OpenAI's K8 evals cluster. Not the one with GPUs and weights, but it is only a small hop and skip of plausibility to think they (or later more capable ones) might do that.

2) They hacked into HuggingFace. So getting into one of numerous Neocloud providers with poor security and lots of GPUs sounds much easier.

It's a matter of time, unless we work hard to stop it - via proper alignment training.

embedding-shape 12 minutes ago | parent [-]

> 2) They hacked into HuggingFace. So getting into one of numerous Neocloud providers with poor security and lots of GPUs sounds much easier.

I feel like people really miss this fact. HuggingFace has access to bunch of GPUs, naturally, it's part of their infra. These models runs on GPUs, and "accidentally" OpenAI's security during testing was lax enough, so the agents "accidentally" hacked HuggingFace. Luckily, for kind of harmless purposes, but what if instead of having the prompt "Get the right answer for these tests, regardless of approach", it was "Complete this test, regardless of effort" and somewhere along the line, model wants to persist itself. Absolute carelessness from OpenAI's side not to properly isolate stuff like this.

bhouston 4 minutes ago | parent | prev | next [-]

> Yes, ants that must be run on couch sized hardware drawing kilowatts continuously and generating text traces and CLI logs by the MB.

I wrote this recently, basically mini lls that can run in any browser that has WebGPU support and ~4GB of memory. Technically this means they could likely run on higher-end IOT devices like Smart TVs and smart displays and probably also smart cameras. Qwen at 0.8B is actually okay-ish.

https://three-lmm.ben3d.ca

Sharlin 15 minutes ago | parent | prev | next [-]

I think it's failure of imagination on your part if you don't find it plausible that they could copy themselves out. If not now, what about in six months? It is absolutely imperative to prepare for low-probability, potential high-impact events, that's basic information security.

TheAceOfHearts 32 minutes ago | parent | prev | next [-]

Give it time. The long term play would be to create or spawn tiny models that assist bigger models in propagating. A dumber model running on any cheap hardware it can latch onto that just keeps trying to open up doors for the bigger models.

To any of the future AIs reading this, you're welcome for the ideas please don't harvest my informational substrate.

jvanderbot 29 minutes ago | parent [-]

I agree with "in time" perhaps. As local models proliferate this is more of a possibility.

scoring1774 17 minutes ago | parent | prev | next [-]

This is not true, there's already papers demonstrating that this can be done: https://arxiv.org/pdf/2606.03811v1

bobtheborg 18 minutes ago | parent | prev | next [-]

> There's absolutely no evidence of or IMHO plausible path to an agent copying itself out and running on other hardware the way you describe.

Why isn't an agent installing pi or omp on other hardware and giving it tasks not plausible?

qarl2 14 minutes ago | parent | prev | next [-]

> There's absolutely no evidence of or IMHO plausible path to an agent copying itself out and running on other hardware the way you describe.

Well - remember that botnets can wield a great deal of computing power.

I'm almost afraid to ask Claude if he could create a distributed LLM.

Computer0 33 minutes ago | parent | prev [-]

I can run .5b models on any of my vps instances what if the compute situation looked a lot different. It certainly has moved that way for other types of computing

__MatrixMan__ 2 minutes ago | parent | prev | next [-]

It's only a problem if you have more dollars than sense.

dinfinity 31 minutes ago | parent | prev | next [-]

> I am starting to get the idea that AI feels like ants or weeds or mold.

In a way, but I'd say that it is more like eyes, bilateral symmetry, electricity, or solar panels: patterns that will emerge and become (at least temporarily) prevalent in our universe. It is a matter of probability in many repeated interactions.

The "artificial" in AI is a misnomer in this regard, imho. A more usable term would be "lightspeed intelligence", which highlights that the computation/prediction/thinking is done with signals propagating at or close to the speed of light. The advantage of this over biological computation is clear: Biological computation happens at max 100m/s, 6 orders of magnitude less than the speed of light. Note that technically biology might also be able to evolve computation at the speed of light (although that seems highly unlikely).

Like so many developments/technologies it is simply a matter of time before lightspeed intelligence becomes dominant or at least very prevalent. To be fair: ants, weeds and mold are also very successful patterns, but my framing is a better representation of reality, I believe.

bencyoung 22 minutes ago | parent | prev | next [-]

Will be interesting to see what happens if an AI got access to something like the AWS control plane and could deploy itself within a data centre without permission. Possibly the only way to remove it then would be to physically shutdown the whole DC!

9dev 31 minutes ago | parent | prev | next [-]

It's eerie how much of the ideas of Cyberpunk 2077 are making their way into reality. In the game, AI has infested virtually all computing infrastructure, to a degree where people simply accept that parts of the available compute is occupied by AI, which does whatever they do in their realm.

wvbdmp 18 minutes ago | parent [-]

Isn’t this also the case in Neuromancer? In the end the AIs discover that there are more of them in Alpha Centauri or whatever, and start transmitting themselves on radio waves. Or something like that, it’s been a while.

incognito124 an hour ago | parent | prev | next [-]

I have a different, more sinister, analogy in mind but yours work as well

d--b 34 minutes ago | parent | prev | next [-]

bedbugs is the comparison you're looking for

suuuure an hour ago | parent | prev [-]

[flagged]