So people here are likely rushing to the comments to ask how can I defend against this?

Bad news I'm afraid, for now aside from Huggingface trying to stop these, you have to treat the AI like a hostile employee.

- Don't give your AI secrets access, scope it's fs access as a limited user.

- Lock down your AI machine's network access to trusted hosts at its egress point on another machine (unfortunately this probably means losing GitHub).

- Filter your AI requests via another, different reputable model from another company / dev.

- Limit the times your AI has access to the internet, it's only allowed while you're actively using it to reduce likelihood of triggering and date/time based exploits (this of course causes issues with a lot of use cases like long running reverse engineering).

I think this could justify a fast, known reputable/good, 2B model acting in a reverse proxy capacity inside the harness and on your wider network.

That model would strip / rewrite data from all requests that looks suspect (it won't prevent data smuggling, like in image uploads).

It would also block domains BAU for a proxy using an allowlist.

I'm not a security expert by any means, so if you know of more defences please reply :)