> They really do come out looking totally incompetent.

Yeah, this is my take away, they should be straight up disallowed from running further testing like this. Clearly they had nowhere close to enough isolation, ran all this on 3rd party infrastructure even though same stuff happened in the past years ago, and even now it's clear the agents successfully broke out just days before?? Really embarrassing stuff, and scary that these are the people supposedly sitting and are responsible for some of the most powerful LLMs on the planet...

What is astonishing to me is how little attention is given to the difference between the original, university-developed ExploitGym and the OpenAI environment. In ExploitGym, the sandbox was enforced by a network-layer policy. OpenAI downgraded that to trusting the repository server's settings. That's a foreseeable problem. Then to actually see the agents exploit it and still not upgrade the isolation to basic best practice? When the best practice was actually listed in the original benchmark and you changed it?