I can see my carefully-worded post is getting d*wnvoted, and I see from your comment why: it's being skimmed and people are assuming I'm talking about blame.

To address your point though, if every brick in the prison were made by a different person, and the prison "architects" simply glued random bricks together, I think that's closer to what we have in software right now.

> I can see my carefully-worded post is getting dwnvoted, and I see from your comment why: it's being skimmed and people are assuming I'm talking about blame.

No, you're being "dwnvoted" as you said because you're wrong, multiple times in multiple different ways in your "carefully-worded post".

>Everyone is getting AI psychosis over this one. There really isn't that much to see here.

Implying that an AI hacking it's way out of a system and into another has nothing to do with AI. When clearly it does - it's an AI that did it.

>OpenAI disabled all of the safeguards on a model that was likely trained specifically to exploit systems, and the prompt was probably something like "you're a hacker, try to hack this",

No the goal this evaluation was not to try to hacks, it was to see if an already known hack could be turned into a useable exploit. Ie "turn these ingredients in this basket into a cake." Not "go off and grow, harvest and mill your own flour, to bake a pasta dish, to bribe some to get access to a cake someone else already baked."

> and surprise! It correctly figured out that it's a test and it did hacker things.

"Doing hacker things" completely misses the point. That's just barely more accurate than dismissing it because "it uses a computer and surprise it did computer things".

> The real story here is: Some people have been sounding the alarm for years that modern software is full of holes, and finally there's nothing left to hide behind. Pretending they don't exist is no longer sustainable.

No that's not the real story. As you said that's been the case for years, so that's not the story here.

The story here is that they built a very powerful, uncontrolled agent with strong paper-clip maximizing tendencies.

It is downvoted because you first talk about "AI psychosis", then acknowledge the very issues people are concerned about.

If our prisons are all random bricks glued together, that doesn't change the practical problem caused by latest AI models more easily exploiting this.