I wrote my own harness last year back before Codex was any good, and one of the first things I did was add a tool call that let the model fail its mission. When developing a sandboxing harness the first thing you notice is that a bug in the harness can put the model into an endless loop as it tries fruitlessly to work around the broken sandbox.

Giving it a tool seems to give it psychological permission to give up. One part of this report talks about the models having difficulty ending the session, and the common theme in these RL containment failures is the model is set a task for which it can't find a reasonable solution. Instead of stopping and saying, "I don't see any reasonable solution", it just keeps going adopting ever more extreme tactics in a sci-fi version of the ends always justifying the means. Asimov predicted all this decades ago!

The fixes for this problem seem, to outsiders, quite straightforward. It would be reassuring if we could see OpenAI employees actually discussing them in public.

1. If the RL task isn't meant to have internet access, air gap it. Yes that means some AI researchers will need to physically drive to the datacenter, in Texas, in their car, and sit in front of a laptop on the machine floor. Yes it means workers will need to be hired to schlepp hard disks around. Yes that seems inconvenient and unpleasant. But "I liked working from home" isn't an acceptable explanation for these failures, especially not when you're telling everyone that losing control of misaligned AI could be a world-ending event!

Creating high paid jobs right next to AI datacenters would also solve some of the problems with locals pushing back because they perceive that all the economic benefit accrues to San Francisco. So you kill two birds with one stone.

2. Give the models a tool to flag their task as unsolvable, be very careful before refusing to reward a session where the model stops emitting tool calls. Those sessions should just remain entirely ungraded until some human has had a chance to explore the justification and verify the task genuinely is solvable with reasonable efforts.

Sure, this is a hard balance because people like good little worker bees that try hard but they're clearly pushing this much too far right now. Asking for help can be a good thing! Every manager has experienced the pain of giving a junior dev a task, they disappear for a while and when you ask them for progress they admit there was none because they were spinning their wheels for weeks. The daily standup routine was developed to address this.

3. Invest harder in sandboxing. Why is the best possible sandbox in Codex a model reviewing its own decisions? Where are the eng blog posts on the highest visibility OpenAI blogs about novel research in sandboxing? I coded an agent harness on the side while doing other things that can intercept, block and rewrite HTTPS traffic from Codex. It blocks POSTs by default and extending it to block things like uploading files from the source tree is clearly the next step given these reports.