> There's no reason that stronger network protections would need to be turned off for a research model that's not supposed to have internet access.

They didn't turn off the network protections - it looks like they used the same system as other sandboxed agents, which is to block all network access except for an HTTP proxy which only allows read traffic to PyPI and a few other package installation targets.

I'm hoping a more detailed retrospective clarifies if the vulnerability the model found in that proxy affected their other production systems as well. It sounds like a data exfiltration mechanism, and I have a keen interest in those.

My current hunch is their biggest error was not using the same network monitoring on their research machines that they use in production. Their network monitoring really should have spotted what was happening as soon as the model broke out.

(It's also possible they were running the eval on a developer laptop somewhere!)

> They didn't turn off the network protections

Right, but this is my point in saying I can't reach the same conclusion that they've invested a lot into these systems for production models. Either they had better network protections, and turned them off in their "sandboxed testing environment" (their words), or they don't have more comprehensive network protections at all and rely on this one, extremely thin layer even in production.