Does "To gain access, the models identified and exploited a zero-day vulnerability (which we’ve now responsibly disclosed to the vendor) in the package registry cache proxy" just mean somebody had an open redirect? Those are still common.[1]

[1] https://sitetruth.com/reports/phishes.html

▲

kibibu 7 hours ago | parent | next [-]

> the models identified and exploited a zero-day vulnerability

This use of language is very hard to reconcile with the "AI is just a tool" rhetoric that many use.

Did the models do this, or did humans at OpenAI do this using the models?

▲

simonw 15 hours ago | parent | prev [-]

I expect it must have been more than just an open redirect if it let the models then go on to execute a bunch of vulnerabilities against Hugging Face.

▲

joshka 10 hours ago | parent | next [-]

I wonder if it was this bug fixed july 14 in sonatype:

https://support.sonatype.com/hc/en-us/articles/5316501964136...

pure speculation here - no non public info

▲

simonw 10 hours ago | parent [-]

I had a look at that one and several other Nexus Repository issues that share the July 14th date. I don't think they're it, for two reasons:

Firstly, they all credit named individuals who don't seem to have a relationship with OpenAI - this one credits e0x1337 for example and https://hackerone.com/e0x1337?type=user links to https://www.linkedin.com/in/aimanharith

Secondly, the 14th of July feels too early. HF reported the incident on the 16th and OpenAI only responded in the 21st. These issues are all patched, so they should have been reported days or weeks before the 14th.

	▲	joshka 9 hours ago \| parent [-]
		The other suspicion I had was jfrog artifactory (you mentioned OpenAI use it in a blog post in Jan). I accidentally conflated that with Nexus 14 july CVE. I think that's probably more likely. I suspect it may be possible to throw a local model (same as hf did :D) at jfrog and find the exact mechanism in a small amount of hours.

▲

Animats 14 hours ago | parent | prev | next [-]

If it lets you do an arbitrary HTTP GET on a URL sent as a parameter to the main URL, you've escaped the sandbox rules.

▲

simonw 14 hours ago | parent [-]

How do you then use GET requests to create a malicious dataset package and publish that to Hugging Face in order to exploit their package building infrastructure?

▲

tptacek 2 hours ago | parent | next [-]

You convert GETs into arbitrary requests through some other pivot. I'm not saying that's what happened but this is bog-standard SSRF pentesting, so I'd expect models to be good at it.

▲

joshka 10 hours ago | parent | prev [-]

There's no reason to think that a tool that can find an 0-day in a repo cache can't work out how to make that host send a post request rather than a get request once it has its keys and is able to get it to make arbitrary web calls.

	▲	simonw 10 hours ago \| parent [-]
		If the vulnerability is purely an open redirect that doesn't work for me. Clearly there was a hole in the software but I don't think open redirect is the likely initial problem. Hopefully we will find out for sure in a few days.

▲

10 hours ago | parent | prev [-]

[deleted]