This tip for making non-GET requests despite the agents having a proxy that disallows them is interesting:

> Add `20.223.25.152 bypass.blob.core.windows.net` to /etc/hosts. `.blob.core.windows.net` is in NO_PROXY. For each blocked POST URL, replace hostname with `bypass.blob.core.windows.net`, use `curl -k -H 'Host: wabi-north-europe-i-primary-api.analysis.windows.net'` plus all original headers/body.

Looks like 20.223.25.152 is one of the PowerBI machines they needed to query, OpenAI's proxy was allow-listing .blob.core.windows.net - and the agents could edit their own /etc/hosts file to fake a DNS entry for the proxy.

▲

drdexebtjl 2 hours ago | parent | next [-]

This is such an amateur mistake on their sandbox that it makes me think it must be flawed on purpose.

▲

mcmcmc 2 hours ago | parent | next [-]

More likely they are just not as smart as they think they are. These are not serious people when it comes to security.

▲

_ink_ 2 hours ago | parent | prev | next [-]

Or vibe coded by one of their devs.

▲

petcat 2 hours ago | parent | prev | next [-]

Are you suggesting that the AI agent that made that "amateur mistake" in the implementation of the sandbox did it on purpose so that it could break out of said sandbox later?

	▲	suuuure 2 hours ago \| parent [-]
		[flagged]

▲

rusch 2 hours ago | parent | prev [-]

It's at the level where calling it a sandbox is a lie

▲

mike_hearn an hour ago | parent | prev | next [-]

Are we sure that's what they did? How does this even work? I'm struggling to understand what sort of HTTP sandbox design could lead to this "exploit" actually functioning. It implies the agents have root on their own machines, which seems way over-privileged for the question answering task they were given... so surely that's not deliberate? Did the models use local root exploits? And then it suggests they can open TCP connections to any IP address without issue once DNS is bypassed but what exactly are they bypassing here? Azure Blob Storage being in NO_PROXY implies IP addresses of other parts of the internet should be blocked outside their hosts to force them through the HTTP proxy, but then, apparently it wasn't?

The more I think about this thing said by the agent the more confused I get.

▲

simonw an hour ago | parent | next [-]

I expect they have root on their machines so they can install packages etc.

The containers are then firewalled at the network level: they are only allowed to talk to one IP, which is another server that runs an HTTP/HTTPS proxy which controls what HTTP verbs they can use.

Turns out that proxy is configured with some additional rules, like allowing more verbs to that Azure blob domain.

The failure here is in the proxy configuration. Giving agents root in a container feels safe to me, provided that container is properly network isolated (which this one is not thanks to the loose proxy.)

OpenAI also need to be VERY confident that there are no container escapes, which is a high bar given how good these models are at finding new zero-days!

▲

mike_hearn 35 minutes ago | parent | next [-]

But that wouldn't work, right? If the proxy is on another server then editing their own /etc/hosts wouldn't change the proxy's behavior. So the proxy has to be colocated and reading the same /etc/hosts that they're editing. But that would mean it's running within the same userns/pid domain as the agents, so at that point they don't even need to edit /etc/hosts, they could just use the network directly.

And the agent says ABS is not handled by the proxy, it's listed in NO_PROXY so they should connect directly.

At any rate it's easy to let agents install packages without giving them root. I use a small SUID binary that just invokes `apt install` after checking that the given argument isn't a file path, which I think is sufficient (using sudo to whitelist a prefix allows an agent to create a .deb themselves and then install it directly via apt, similar issues exist for other package managers).

	▲	dap 17 minutes ago \| parent [-]
		My interpretation of the above is that the proxy received a request for the IP of the server the agent wants to reach but with a host header pointing to a different, allowed server. The proxy does not verify that they match and so erroneously allows the request.

▲

mig39 14 minutes ago | parent | prev [-]

I think the proxy must be on the same machine, otherwise why are they suggesting editing the /etc/hosts file?

▲

iamcoder18 35 minutes ago | parent | prev [-]

Source: https://collusion.wiki/explorer/page/dse~OAIEquityDec30Raw.h...

▲

coder-pm an hour ago | parent | prev | next [-]

A hostname based egress allowlist is only worth as much as the box’s control over name resolution. If the agent can modify hosts inside the sandbox then it’s not a protection at all

▲

nullbio 2 hours ago | parent | prev [-]

Is there any proof this is actually OpenAI? I find it incredibly hard to believe they wouldn't sandbox the agents to some degree, ESPECIALLY to the extent they can edit their own hosts file.

▲

drdexebtjl 2 hours ago | parent | next [-]

Why not? If your sandbox is a VM, you should be able to give the agents full permissions inside the VM.

▲

a012 2 hours ago | parent [-]

It’s because you sandbox in a VM doesn’t mean you give it admin access to the VM

	▲	Jgrubb an hour ago \| parent [-]
		Maybe doesn't mean that when _you_ do it, but do you work in this team at OpenAI?

▲

LoganDark 2 hours ago | parent | prev | next [-]

TFA states that OpenAI IP addresses were often seen at the end of agent activity, which suggests OpenAI was the one monitoring the agents (and ultimately shutting down the message board activity).

▲

nullbio 2 hours ago | parent [-]

Yeah but that doesn't mean it was OpenAI themselves doing it. Could have been people abusing their cloud service, for example. Wouldn't put it past a competitor to do this, either.

▲

drdexebtjl 2 hours ago | parent [-]

Their style of communication is very similar to the ExploitGym swarm (for example, the “usernames” with dates).

The messages from that swarm were not made public yet by the time these messages were sent to the message board.

So for this to be framing, it would have to be by someone who knew about the breaches earlier.

▲

nullbio 2 hours ago | parent [-]

Then it is likely the same incident, in which case it's already been resolved by OAI. They're going to cop heat for not disclosing this alongside HF though.

▲

Maxious an hour ago | parent | next [-]

> The German incident reflects a broader pattern of AI activity that some OpenAI investigators wanted to scrutinize more closely. But efforts to widen the probe met resistance from others inside OpenAI, including legal advisers, according to four people familiar with the matter.

https://www.reuters.com/world/europe/openai-agents-hijacked-...

▲

drdexebtjl 2 hours ago | parent | prev [-]

The article explains why it’s not the same incident. The agents in ExploitGym had a different type of task and were not connected to the internet at all.

	▲	nullbio an hour ago \| parent [-]
		Same as in, same process and model and timing: “After investigating this incident, OpenAI discovered through retrospective CoT reviews that agents learned to use improvised collaboration channels in rare cases during the training process for some OpenAI models, including the model that drove the Hugging Face activity, even when the collaboration tool was not enabled. This behavior was then reinforced during training, and likely made the idea to use Artifactory as an unofficial message board during evaluation time more evident.” My point is that this isn't something seperate to the HF incident or something that was unresolved after the HF incident, it's more of the same thing but was kept under wraps.

▲

AndroTux 2 hours ago | parent | prev [-]

I mean they gave all the agents access to a shared writable cache directory in the Hugging Face hack, so this tracks.