Remix.run Logo
jvanderbot 20 hours ago

There's one flaw in the evidence for the logic chain. The hugging face attack is used to demonstrate three things: the need for oversight today (fundamental to the article) and to demonstrate some kind of drift or unexpected capability gain, and finally to hint and some fundamental morality of the AI or at least the risk of drift from our morality.

I'd argue that what the hugging face attack illustrates is that large AI companies are motivated to have bombastic claims supported by bombastic demos. The model was clearly trained or encouraged to work as it did, as evidenced by the fact it keeps using this particular escape hatch.

And the fact that it aligns with prior and current calls for what very likely might be a regulatory capture / oversight capture move right before IPO. It aligns so well with this "barely constrained superweapon" narrative it might as well be PR.

dist-epoch 19 hours ago | parent [-]

How does your logic explain that OpenAI seems to want to hide the extent of the HuggingFace incident, and every week we learn from 3rd parties about new victims of the hack?

4D chess? They want others to find the hacked services, so the report of how dangerous the agents are seems more "legit"?

jvanderbot 18 hours ago | parent [-]

IMHO hiding details of the hack helps conceal that they built it to do what it did, and maybe were able to know it was working just as intended.

The fact it keeps doing it, with more and more evidence, is a sign that it's built that way.

This is a program running on their montoroed machines that they purpose built and monitored its training at every step. I think it'd be way more suprising that they didn't know it used note taking and cross-run memory.