> I expect OpenAI would agree with you that "the lack of a real sandbox or airgap is nothing short of irresponsible on OpenAI's part". They have clearly invested a lot in those systems for their production models, but in this case they had deliberately turned a bunch of them off for a research project.

I'm not sure I can reach the same conclusion. There's no reason that stronger network protections would need to be turned off for a research model that's not supposed to have internet access. There's no reason that it they couldn't continue to run in whatever hypervisor or VM that any other production systems could use. The only thing that should be turned off for this type of work are the non-deterministic classifiers that they use for auto-approval mode in their harness, and I would wager that's the only one they did turn off.

> There's no reason that stronger network protections would need to be turned off for a research model that's not supposed to have internet access.

They didn't turn off the network protections - it looks like they used the same system as other sandboxed agents, which is to block all network access except for an HTTP proxy which only allows read traffic to PyPI and a few other package installation targets.

I'm hoping a more detailed retrospective clarifies if the vulnerability the model found in that proxy affected their other production systems as well. It sounds like a data exfiltration mechanism, and I have a keen interest in those.

My current hunch is their biggest error was not using the same network monitoring on their research machines that they use in production. Their network monitoring really should have spotted what was happening as soon as the model broke out.

(It's also possible they were running the eval on a developer laptop somewhere!)