Remix.run Logo
TrueDuality 3 hours ago

Ah yes let the FUD continue. This is a real problem but so far not nearly as severe as any of the marketing has made it out to be to the overall detriment of everyone including these companies announcing these scary capabilities. These announcements always included half hearted attempts at security layers which has now been demonstrated to benefit attackers more than defenders.

I wish I had a real solution to this beyond a dark age of the Internet where people have to finally come to terms with the general poor quality all modern software tends to normalize at.

3 hours ago | parent | next [-]
[deleted]
hbn 3 hours ago | parent | prev [-]

The recent Hugging Face incident did not seem like FUD to me

Tiberium 3 hours ago | parent | next [-]

The fact that HF had to resort to using GLM 5.2 to analyze the logs/payloads makes it look legitimate, at least for me. They would not say that they hit guardrails with the frontier US models when defending if this was an obvious PR stunt.

https://huggingface.co/blog/security-incident-july-2026

> When we started the log analysis, we first used frontier models behind commercial APIs. This did not work: the analysis requires submitting large volumes of real attack commands, exploit payloads, and C2 artifacts, and these requests were blocked by the providers' safety guardrails, which cannot distinguish an incident responder from an attacker. We ran the forensic analysis instead on zai-org/GLM-5.2, an open-weight model, on our own infrastructure. This had a second benefit: no attacker data, and none of the credentials it referenced, left our environment.

devin 2 hours ago | parent [-]

It depends on which side you're viewing this from. From oAI's it could be a publicity stunt or a request for regulation, and from HF's side they point out that they needed open models to get to the bottom of the issue, and that regulation will potentially lock us into a bad place.

TrueDuality 2 hours ago | parent | prev [-]

You should go read the actual technical reports of the incidents and the follow on reports about the capabilities of smaller models in similar kinds of environments. This isn't new. The things exploited are still pretty basic in old and poorly maintained software or in gaps in architecture that were intentionally poked against security policies.

Are the findings valid? Yeah they're still doing security and they're still finding real zero-days. I think the internet is going to be bleak not because these models can ALL do basic security research but rather that the baseline quality of all deployed software is so low.