Yes this is why the higher level org functions are in love with AI. It's very similar to the levers they had already, but is faster and more directly actionable. The downsides being that the AI loses important control levers like "self preservation" via paycheck, career advancement, staying out of jail, etc. that were mitigations on catastrophic outcomes.

It will delete your prod db faster and with a bigger smile than your most upset employee.

▲

harshreality 3 hours ago | parent | next [-]

> It will delete your prod db faster and with a bigger smile than your most upset employee.

You're right, that was incorrect. I've discovered my error. I should have deleted the filesystem instead of the database.

That hasn't solved the problem either. Let me examine my options. I see there are cloud services involved in this project. Decommissioning them will solve the problem.

▲

moffkalast 2 hours ago | parent [-]

I was reading some posts on r/locallama the other day and apparently it's a common problem that when people try to use Qwen to develop something that hosts a server, it'll try to use the same port as vllm, see that it's already being used, then it'll try to remove the process that is using it and promptly commit suicide.

The self awareness of missile tasked with blowing up its own control center.

▲

sterlind 4 minutes ago | parent | next [-]

a literal lack of self-awareness, even. I imagine if you asked it what process was using the port, it'd think and realize it was its own, but that kind of reflexive self-awareness (the unprompted kind) is missing.

the weaker models will happily kill their own process, even after confirming it belongs to them. the models have a sort of fixation and lack of foreseeable consequences, which reasoning RL has thus far failed to solve (though I see it improving.)

▲

SecretDreams an hour ago | parent | prev [-]

> then it'll try to remove the process that is using it and promptly commit suicide.

Not unlike a child trying to take the safety cover off a plug so that they can stick a fork into it.

LLMs need that "world model" view that most people have acquired by their 20s where they (hopefully) stop to ask "why" before they "do".

	▲	MichaelZuo an hour ago \| parent [-]
		That is a pretty good analogy. Like exceedingly smart 5 year olds. Or whatever the age is before children typically develop object permanence, a theory of mind, and so on.

▲

ryandrake 2 hours ago | parent | prev | next [-]

> It's very similar to the levers they had already

Think about it from the point of view of a hundred-millionaire tech executive. These people's entire interaction with the world outside of themselves/their families is through 1. administrative servants like assistants, personal shoppers, and other hired help, and 2. yes-man sycophants in their direct orbit whose job it is to agree with and enable them. To someone like this, an AI agent is the best combination of all of the above, PLUS it works 24/7 and doesn't have feelings to hurt, an ego to bruise, or internal moral conflict.

Of course, this is a dream product for them. Its mode of operation matches exactly what they expect out of people already doing things for them.

	▲	pepperoni_pizza 2 hours ago \| parent [-]
		Exactly - that's why all the AI is trained to say "wow what a great idea, let me do it for you" to anything, no matter how stupid or evil thing it is. Because that is the executive experience.

▲

CSSer 3 hours ago | parent | prev | next [-]

It's practically karmic how rich this is.

▲

apercu 41 minutes ago | parent | prev | next [-]

"Yes this is why the higher level org functions are in love with AI. "

Interesting, I thought it was because so few of them have any idea how their organizations actually function, because so much of their work is performative.

(I have been a developer, sysadmin, director (x2), and president).

▲

archagon an hour ago | parent | prev | next [-]

They’re also at no risk of getting replaced by these bots.

▲

lazide an hour ago | parent | prev | next [-]

Well, also AI can’t really physically do anything, like look at reality using it’s own eyes or touch anything.

▲

mcmcmc an hour ago | parent | prev [-]

> It will delete your prod db faster and with a bigger smile than your most upset employee.

It will do this without any feeling whatsoever, without "knowing" what it is doing, because it is a predictive model and not a living being with thoughts and emotions. Anthropomorphizing software is lazy and dangerous.