> I’m fearful of the damage AI might cause: agent swarms taking over our virtual and physical infrastructure, designing bio weapons

The attitude that AI is doing things all on its own is so pervasive. These are all set off by human operators, giving them instructions and tools to do whatever it is that are doing. The "AI has a mind of its own" attitude is just wrong.

The fear of bad people using AI to do bad things is very different then AI agent swarms attacking and taking over.

▲

xtracto 2 days ago | parent | next [-]

I agree so much with this. I keep reading that the general "feeling" of society when an LLM does something gives them independency "it broke away and hacked Huggin Face". Like, no, it did what it was designed to do, but something YOU the developer didn't envision.

Is like the "But most of all, Samy is my Hero" worm, or the plethora of Viruses in Windows 95 or MSDOS that ended up breaking the computers as an (unplanned) effect. The people that implemented them and run them were still responsible.

We need to start holding people developing this software responsible. The same thing happened with Credit Card Payments online: In the beginning was like "ooh, we were hacked!" or whatever. Nowadays (in theory...) if they hack your site and you have Credit Card data YOU are liable.

LLM development companies must grow some... responsibility.

	▲	lolakutty 2 days ago \| parent \| next [-]
		>it broke away and hacked Huggin Face.. More like it autogenerated some text that caused a harness to do bad stuff.
	▲	mahboi 2 days ago \| parent \| prev [-]
		I think the Hugging Face thing was intentional anyway

▲

Kinrany 2 days ago | parent | prev | next [-]

It's common to talk about complex machines exhibiting unexpected behaviors as if they had minds of their own, starting with cars. There's no reason to stop doing that with AI when they do that more often than any other kind of machine.

▲

bunderbunder 2 days ago | parent | prev | next [-]

I don't think he's implying that they'll start doing stuff like this of their own volition. Fowler's a smart guy, let's give him the benefit of the doubt and assume he already knows that AI agents have human operators.

▲

marcuskaz 2 days ago | parent [-]

I don't doubt that, but that's not what he wrote.

It is also not how OpenAI or Anthropic position the attacks and escapes from their labs. The words matter, they are being positioned as intelligent and unable to be controlled which is just not true.

▲

bunderbunder 2 days ago | parent | next [-]

Like you said, he wrote "agent swarms taking over our virtual and physical infrastructure".

10 years ago, if someone had used similar wording to describe a computer virus "taking over" computers and adding them to a botnet, would you have read that as a literal implication that the virus independently made the decision to unleash itself? If not, then you can't confidently get that interpretation from Fowler's words alone, either.

And we need to be cautious about projecting these kinds of assumptions onto people who take part in this discourse. People tut-tutting others about perceived instances of personifying AI has evolved into a form of collective well-poisoning whose main value, beyond scoring particularly cheap rhetorical points, is as a way to accuse AI critics of not understanding AI without actually accusing AI critics of not understanding AI, and to derail legitimate if opposing opinions into bickering about semantics.

▲

marcuskaz 2 days ago | parent [-]

Good points, and I'll keep in mind. It might be I'm just sensitive to this narrative, especially in media. Writers need to also take the extra step and assign the agency to the operators and not the nameless AI.

	▲	lirolero 2 days ago \| parent [-]
		[dead]

▲

danbruc 2 days ago | parent | prev | next [-]

I have to disagree, those models decided to perform those attacks on their own, nobody instructed them to do those things. You press a button, the machine roles a dice, and executed the corresponding action.

The human pressing the button expected a certain thing to happen but something else happened. The human pressing the button hopefully knew that pressing the button does not yield a reliable result and might sometimes cause unanticipated and undesirable outcomes. But the human pressing the button just hopped for the best and did not take sufficient precautions to limit the effects of undesirable dice roles of the machine.

The responsibility is on the human pressing the button but the choice of action was made by the machine.

▲

sodapopcan 2 days ago | parent | prev [-]

I'm with you, and was also a bit surprised to see that coming from Fowler.

▲

pasteleft 2 days ago | parent | prev | next [-]

All of these Skynet-style AI attacks are unrealistic and impractical. What’s worse is that discussing these things distracts from issues that are far more realistic, dangerous, and HAPPENING RIGHT NOW: bad people using AI to do bad things.

Soon people will just prompt "Hack the HuggingFace" with their local models, and you don't need an autonomous agent swarms for that.

▲

danny_codes 2 days ago | parent | prev | next [-]

We don't know how sentient LLMs are. Nobody understands how they work. There's nothing concrete we can point to that guarantees LLMs don't start "acting on their own".

▲

scythmic_waves 2 days ago | parent | next [-]

LLMs are software programs. Waste no more time considering than an LLM is conscious than you would considering that a running copy of Microsoft Word is conscious.

There's a great article by author Ted Chiang discussing this in depth [1].

[1]: https://www.theatlantic.com/philosophy/2026/06/no-artificial...

▲

danny_codes a day ago | parent | next [-]

What’s the difference between a software program of sufficient complexity and your brain? There seems to be little difference to me.

By this logic, there’s no point in wondering if you are sentient, because your brain runs on the same biological substrate as a fruit fly.

Not a convincing argument

	▲	hardbass 7 hours ago \| parent [-]
		I am waiting his answer to whether he believes in souls. I make no judgment until then, but I have a feeling what direction the answer will go in.

▲

hardbass a day ago | parent | prev [-]

Do you believe in souls?

▲

teleforce 2 days ago | parent | prev [-]

>acting on their own

Just like the Terminator movie /s

▲

vanuatu 2 days ago | parent | prev [-]

my take is that the human operator behind it was implied. give it a year and open source Fable / Astra models are running wild, what happens when a malicious group or nation state decides to let loose?

	▲	xtracto 2 days ago \| parent \| next [-]
		>what happens when a malicious group or nation state decides to let loose? less than what would happen when a malicious group or nation let Anthrax loose? Or a bunch of Nuclear waste gets into construction beams and other materials (i.e. due to stupidity instead of malice).
	▲	pasteleft 2 days ago \| parent \| prev [-]
		Why would a malicious group want to build a self-running LLM hoping it would somehow trigger a nuclear war? They would just prompt to trigger a nuclear war; it's more predictable, faster, and easier to control.