You know this makes OpenAI look really bad, right?

Hugging Face had to tell all of their users, many of them paying customers:

> As a precaution, we recommend rotating any access tokens and reviewing recent activity on your account. If you believe you are affected, or want to report a security concern, contact us at security@huggingface.co.

HF also said this, I'd be very interested to hear how that got resolved!

> Finally, we have also reported this incident to law enforcement agencies.

▲

cayley_graph 15 hours ago | parent | next [-]

I could fully see them thinking the incident disclosed yesterday would have made them look good ("wow, OpenAI's models are so capable!"). That it didn't occur to them to discuss specific preventative measures to be taken in the future (airgapping as a foolproof one already familiar to the CTF world, anyone?) indicates to me they're not taking their job seriously; they are the ones treating this as a marketing charade.

It's very difficult for me to reconcile belief in the existential risk business with what they actually did. So I agree with you that this makes OpenAI look badly incompetent; but their communication on this makes me think they don't realize it.

For what it's worth I don't agree with the xrisk-ness of these models; they're dangerous, but almost certainly only temporarily while a new equilibrium is reached via more secure software. Open models are probably an essential part of the recipe (as you noted) for doing so. I also have a personal suspicion that LM-accelerated formal verification will have no small role to play here, sidestepping the cat-and-mouse game of bug finding-and-fixing.

▲

simonw 15 hours ago | parent | next [-]

Here's the language that makes me think they are taking this seriously:

> We are sharing preliminary findings at this stage to help defenders understand what happened and to help calibrate on what models are now capable of. We will continue to conduct a thorough investigation alongside Hugging Face and will share more details on the vulnerabilities, incident, and findings when our investigation is complete.

That's not well massaged PR language - that's the kind of thing you dash out when you see a major shitstorm brewing (HF had already publicized the attack before they knew it was from OpenAI) and you want to get ahead of things while you're still pulling together the full story.

I expect we'll find out within a few days if OpenAI are going to keep their promise to "share more details on the vulnerabilities, incident, and findings". If they don't do that I'll reassess how I interpret their initial post.

	▲	windexh8er 15 hours ago \| parent [-]
		> We are sharing preliminary findings at this stage to help defenders understand what happened and to help calibrate on what models are now capable of. With who? Who are these "defenders"? None of the US labs have done much for the greater good as of... Ever. Of course a frontier provider can leverage their own resources at scale and pull something like this off. If anything this should showcase how dangerous OpenAI and Anthropic are in their current states and maybe the powers shouldn't be concentrated as they continue to move. I will bet that the RCA debriefed by OAI is going to be a lot of lipstick and very little meat.

▲

skeledrew 13 hours ago | parent | prev [-]

> airgapping as a foolproof one

How would the model get any packages that it thinks it needs to complete the task at hand? Not a well-specified task that those tasking it could anticipate and provide all resources up front, but one of discovery.

▲

14 hours ago | parent | prev | next [-]

[deleted]

▲

15 hours ago | parent | prev | next [-]

[deleted]

▲

protocolture 13 hours ago | parent | prev [-]

>You know this makes OpenAI look really bad, right?

The target audience is regulators. They want to look like the smart guys really concerned about AI safety, when they come asking for open weights models to be banned and for other regulations to cement in their moat.

They want this to look like a demon core incident. Bomb and Nuclear reactors still got built.

	▲	simonw 10 hours ago \| parent [-]
		The lesson OpenAI and Anthropic should have learned from the whole Fable export controls thing should have been "don't pull stunts with the US government". Turns out they can backfire.