> Humans were doing exactly what humans are expected to do when facing advanced AI. Being outmatched.

"Being outmatched" is not a novel situation for humans either individually or collectively and there are a hell of a lot of ways we can approach that situation productively. OpenAI doesn't appear to have bothered.

Here's a freebie: if you're building something that might turn out to be Skynet and you don't know what it's capable of, your testing regime should assume it is capable of doing bad and unexpected things and account for that possibility: airgap if you can, monitor all network traffic, monitor all hardware usage statistics, log everything, constantly analyze logs, collect baselines and snapshots, also don't trust anything from a device that a model is running on without cross-correlating with other information as much as possible (does your AI inference server claim low utilization? put a temperature probe on it and see if it's staying cool or getting hot, maybe Skynet-Alpha is overwriting /proc to mislead you for reasons you don't yet understand!)

In other words, if you WANT to be able to nip things in the bud - buy some nippers and watch for buds. Whatever else this situation is, or may turn out to be, it is not a situation where OpenAI was on their guard and still got surprised.

My cognition is outmatched by predicting the impact of throwing a brick over my neighbor's fence. I have no idea if it will land harmlessly in a patch of grass or fracture her skull. Once I've thrown the brick, even if I see my neighbor enter her yard, my reactions are too slow to save her.

I'm not the wisest man, but I'm wise enough not to throw the brick and see what happens.

Similarly OAI should have the wisdom to see that deploying a hazardous swarm of agents with access to the public internet could result in harms, and that those harms would manifest quicker than humans can react, but they unleashed the swarm anyway.

Super charitable reading imo. This is like saying we can’t detect a speeding car because we can’t run as fast as a fast car.

It’s not like the humans were engaged in some kind of battle of wits with some super AI, it’s just some employee not monitoring the output of an experiment.

All it would have taken is someone to peak at the output tokens during the run and it would have been obviously the test had gone off rails.

I believe that this comment is exactly the intended outcome of this “incident” and these reports.

I implore you to approach these situations with at least a hint of cynicism.

These “advanced foundation models” escaped their “sandbox” and conducted an attack on their own? Meanwhile the highest capability models available to the public still struggle to write a unit test for a codebase larger than a hobby app without large amounts of tailored human guidance.

What is more likely here - are you looking at research on an emergent phenomenon, or are you looking at advertising copy around an engineered scenario from business partners?

I get the snark (and slightly agree), but that's not really what GP or TFA were saying at all. They are saying that these were the least things we could have done.

What you're saying is, "Your scientists were so preoccupied with whether they could, they didn't stop to think if they should"

while the author of the TFA was saying, in effect: "your scientists didn't even bother with the most basic duty of care"

Life finds a way, or, in this case, super-intelligent AI.

Part of me would like to believe that they are also intentionally making models that are good at hacking without safety at all for governments willing to spend billions on them.

In that light you're likely most worried about other people hacking in and stealing the model and information from you. And at the same time you have massive amounts of alerts and data on systems attempting to break out because that's what you want them to do so you train yourself to ignore them.

The way I interpret their statement is if a person spins up an agent and that agent hacks some company/organization/government/etc, then that person is at fault for committing the crime. That "well my agent broke containment and acted on its own" should never be accepted as a reason for the occurrence, and the person who kicked off the agent is responsible for all actions the agent takes.

A registration system would be more for tracing back agents to people, but I agree that is very difficult to actually enforce as a system.

> Plain old US tort law should do the trick.

Difficulty: these companies are run by people (many of whom also read Milton Friedman) and who have participated in the regulatory capture of the justice system. They've convinced lawmakers to put limits on damages. They've put arbitration clauses in their ToS. They've got well-funded legal departments that can outlast a person who has to pay out-of-pocket for a legal team just by filing motions to delay proceedings. Sometimes they'll just file SLAPP suits against people they don't like.

If tort law is to be a remedy, then average people have to feel like there's a chance the remedy will go their way. To make that a reality will take several major reforms at the local, state and federal level that the people with money absolutely will not tolerate.

you are honestly comparing Louisiana florists to OpenAI in order to support "just say no Licensing by government" ?

The alternative is to create a permanent overclass that can hack anyone consequence-free, because they can blame it on AI agents. That also is rather dystopian.

Faced with those alternatives, I want neither. Is there a way for us to get neither?

From the article:

1. Failure to Care or Respond. The biggest holy shit moment, to me, remains that OpenAI on multiple occasions had teams that found out about the message board, knew that agents were in communication, and they disregarded this. The first known warning was in late May. The warning on June 27 was unambiguous. “Responders investigated and linked the observed internal activity to an ExploitGym evaluation using Artifactory as an improvised message board and a network pivot. At this time, the on-call response staff advised that stopping the evaluation run was not required.”

2. Failure to Monitor. The entire time, there was almost no monitoring of the situation, of what these AIs were up to. That’s how this was not caught.

3. Failures of Infrastructure. OpenAI failed at delivering even basic software security, in numerous ways. One example is the models all having access to the same Artifactory instance. There were also other severe failures. Nor did OpenAI seem to be properly testing for such failures.

4. Failures of Alignment. The biggest failure, the one that counts in the end, was that the models were severely misaligned, and I don’t think they appreciate why.

5. Failures of Attribution. OpenAI’s post-mortem essentially blames events on a real and important series of prosaic failures. But solving that won’t get it done.

6. Failures of Environments and Data. Prosaic failures in the RL pipeline absolutely did contribute to this, especially impossible tasks. This is ubiquitous, all of this is always rushed, as Utah Teapot explained this week.

7. Failures of Decision Making. OpenAI’s post mortem does not ask the question of how Mistakes Were Made, at various points.

8. Failures of Culture. None of this would be possible, let alone all of it, without OpenAI having experienced profound failures of safety culture. I see OpenAI responding to some other aspects with swift action, but no sign on this front.