So they found their agents had RCE'd Artifactory once, reported it and got the fix, continued using Artifactory for their sandbox, and left it unmonitored for days despite the earlier exploits? They really do come out looking totally incompetent.

I stress about my agent sandboxes all the time and the only models I run have the default heavy handed guardrails, and I don't leave them running persistently.

Edit: not to mention, why is your first cybergym not your own sandbox??

> They really do come out looking totally incompetent.

Yeah, this is my take away, they should be straight up disallowed from running further testing like this. Clearly they had nowhere close to enough isolation, ran all this on 3rd party infrastructure even though same stuff happened in the past years ago, and even now it's clear the agents successfully broke out just days before?? Really embarrassing stuff, and scary that these are the people supposedly sitting and are responsible for some of the most powerful LLMs on the planet...

Or they wanted it to happen again. Worst case outcome, the US Congress spontaneously grow backbones, spite their sponsors and unite in a bipartisan effort for effective regulation because a few people got hacked? Likelihood low - the hacked companies might even oppose any meaningful AI regulation because it hurts their inflated profit forecasts. More probable outcomes: Useful real-world testing, free headlines about AI breakthroughs, scare USG into providing more free money ("look how scary it is - what if China develops this faster than us?")

Why is Artifactory the only relevant exfiltration point? It's really not the point at all. Watching the complete video the issue is that the models team up and will go to almost any length to cooperate to accomplish what they think their goals are.

It was the first step in a many step process. Like they said this is a watershed moment and it's helpful to not miss the forest for the trees.

> They really do come out looking totally incompetent

These companies are full of the smartest people the world can produce with little room for complacency. They have a clear, proven investment upside to presenting their technology as "too powerful / too dangerous", and now a clear, proven example that there will be no legal consequences (as if anyone didn't already know that).

Why do we keep giving them the benefit of the doubt that they just didn't know any better?

> I stress about my agent sandboxes all the time

Same here, so I ended up moving the whole dev environment (editors, agents, containers) inside a hardened QEMU/KVM VM that reaches the internet but has no route to the host, the LAN, or any other private address. I wrote a script to create such VMs and also verify network containment by scanning outward from inside the guest. Even then, I still don't feel great when running agents unattended.

Write-up in case anyone's curious:

https://karamatli.com/posts/network-isolated-kvm-sandbox-ai-...

An alternative reason would be that they see this behavior so frequently that it didn't really raise to the level of concern.

And all that just to allow internet access for npm and pypi? If you've got the bandwidth and disk space, it's very easy to make an offline mirror of both.

Right? Like I feel like I’m taking crazy pills.

OAI (and now the other OAI companies not wanting to be left out) are running around announcing they started a forest fire through negligence and incompetence and people are like “Wow they used a really neat lighter!”

I get that it’s fashionable to hate big companies but you’re working overtime here. It’s reasonable to assume that a bug was fixed when reported. And if you think your monitoring is 100%, you don’t know what you’re talking about.

If you consider that incompetence, it’s possible that you’re not a very nice person.

[dead]