| ▲ | simonw 5 hours ago |
| I don't interpret it like that at all. This is deeply embarrassing for Anthropic: it turns out they hadn't been keeping a close eye on their models either, and back in April they successfully attacked three different organizations! The hacks weren't particularly impressive either: > [...] using basic techniques, such as exploiting weak passwords and unauthenticated endpoints. It did not find or exploit any complex vulnerabilities [...] |
|
| ▲ | swatcoder 3 hours ago | parent | next [-] |
| > Deeply embarassing What signals are you using for this assessment? Are they indicating embarassment? Do you honestly see their customers being concerned over this? Like lion tamers in a circus, Anthropic and OpenAI thrive on the theatricality of how scary their pets appear and so they play it up by prodding them to growl and snap at chairs and then mug for the audience every time it happens. And to their delight as performers, the audience gasps and cheers each time. They want to make their pet seem the most powerful and unpredictable and they want their audience to believe that they're holding it back from catastrophe but only barely and only because of what unique talent they have. This is not embarassment. |
|
| ▲ | anon373839 2 hours ago | parent | prev | next [-] |
| I know it seems strange that a company would use its own negligence as a publicity gimmick. But take a look at the smug smirk on Sam Altman's face when he's asked if OpenAI might have attacked companies other than HuggingFace. ("I mean there could be, yeah.") https://www.instagram.com/reel/DbZVL8viUD4/ This is not the communication of a CEO whose company was just shown to be incompetent at performing its security research. No, this attention is very much what he wanted. And it does not take a great leap to infer that Anthropic is now using the same playbook. Note that all the headlines are about "rogue AI", and not about operator negligence. Rogue AI is a sexier story, and their media strategists know that's how it will play. |
|
| ▲ | andy99 5 hours ago | parent | prev | next [-] |
| Then maybe just this timing is really unfortunate, I think most people’s first reaction will be that it looks like a “us too” response to the OpenAI/hf thing. |
| |
| ▲ | 3 hours ago | parent | next [-] | | [deleted] | |
| ▲ | jryle70 3 hours ago | parent | prev [-] | | When they did it with Mythos in April the HN crowd said they were bragging, crying wolf. Now they are "us too". It's fun to bash Anthropic, isn't it? | | |
|
|
| ▲ | cbb330 5 hours ago | parent | prev | next [-] |
| so deeply embarrassing that they published an eng blog about it |
| |
| ▲ | simonw 4 hours ago | parent | next [-] | | If they quietly brushed this under the rug - especially given the PyPI malware that was involved - it would be a huge scandal. Disclosure is the only ethical response to this. | | |
| ▲ | avaer 3 hours ago | parent | next [-] | | The target audience of this blog post does not give a damn about PyPI. The affected parties get literally nothing from this post. They already disclosed behind the scenes, that was the ethical part. Writing PR pieces competing to be the most dangerous model around (so give us money!) is the unethical part. | |
| ▲ | cbb330 3 hours ago | parent | prev [-] | | Then disclose to real organizations. Not twitter |
| |
| ▲ | johnfn 4 hours ago | parent | prev | next [-] | | Companies post deeply embarrassing eng blogs all the time. See: every post about downtime or a security incident ever. | | |
| ▲ | sneak 4 hours ago | parent [-] | | Wrong. These are always humblebrags about how good their ability to learn from their mistakes is, and how robust they were before, and how they are even more robust now. The ones that are "deeply embarrassing" simply aren't posted. | | |
| |
| ▲ | ofjcihen 4 hours ago | parent | prev [-] | | Right? 100% this is them trying to make gold out of turds. | | |
| ▲ | solenoid0937 4 hours ago | parent [-] | | There's nothing Anthropic can do to satisfy the HN crowd, is there? If they don't post about this they're bad. If they post about this they're bad. They are not bragging in this article or they would not have called the attacks unsophisticated. | | |
| ▲ | ofjcihen 2 hours ago | parent [-] | | The real lesson here is still the boy who cried wolf. They’ve played this game for years. I have no reason to believe their worries are real now. | | |
| ▲ | solenoid0937 an hour ago | parent [-] | | I see no fear mongering or "boy who cried wolf" in this article. They are admitting to a fairly mundane network misconfiguration and very basic unsophisticated actions taken by Claude thereafter |
|
|
|
|
|
| ▲ | andromaton 2 hours ago | parent | prev | next [-] |
| We've known artificial intelligence will do unexpected things since the 90s and that it can do difficult things since 2025. Pausing worldwide not easy but we do harder things all the time. |
|
| ▲ | rvz 5 hours ago | parent | prev | next [-] |
| > I don't interpret it like that at all. This is deeply embarrassing for Anthropic: it turns out they hadn't been keeping a close eye on their models either, and back in April they successfully attacked three different organizations! This just helps their (Anthropic) argument into persuading the US government into taking action into limiting powerful closed or open-weight models from being released without going through (yet to be defined) regulatory oversight. The only "embarrassing" thing for Anthropic was that there was little to no continuous security monitoring of this since April, and they then decided to do a cybersecurity transcript review only AFTER the incident with OpenAI and Huggingface. |
| |
| ▲ | simonw 4 hours ago | parent | next [-] | | Anthropic know better than anyone else how risky it is to get this current administration upset with you over safety/security concerns. | |
| ▲ | gck1 4 hours ago | parent | prev [-] | | They also gave access to Mythos (the Mythos) to some companies, based on... vibes. Who knows how these companies are using it. If Anthropic can't effectively contain their own models, can the partners? While the rest of us get fallbacks and warnings, not even being able to defend against the attacks they themselves are causing. Do we really have to re-learn all the industry's knowledge the hard way? | | |
| ▲ | htrp 4 hours ago | parent | next [-] | | You also got access to mythos based on how much you spent with anthropic. I think sales guys were bragging about getting their enterprises access | |
| ▲ | jryle70 3 hours ago | parent | prev [-] | | > based on... vibes According to whom? > Do we really have to re-learn all the industry's knowledge the hard way? Yes we do. That's why there is the saying "regulations are written in blood". Especially for LLM, which not too long ago a lot of people on HN dismissed as stochastic parrot and next token generator. | | |
| ▲ | gck1 3 hours ago | parent [-] | | > According to whom? It's very easy to answer this without my help by trying to get access to Mythos. Do you see requirements clearly listed anywhere?Can you even apply? What you'll find is maintainers of large open source projects and analysts' reports with vague statements like - "should follow strict security requirements": "Trinidad also noted that the Anthropic announcement pointed out that each of the 150 new participants, in Anthropic’s phrasing, “will need to meet our security requirements before they gain access.” Trinidad said the security requirement claim doesn’t build confidence, because “nobody knows what those security requirements are.” [1] It's also some random rich companies like Hitachi or Dragos [2] Do you trust that Hitachi and hundreds of other random organizations will be able to contain Mythos and not accidentally attack your project or your bank? I don't. > Yes we do. That's why there is the saying "regulations are written in blood" We absolutely don't. We have already learned with blood that gating access to security based on the number of zeroes in bank account and authority is a horrible model. We can apply this knowledge to LLMs, we don't have to spill blood again. [1] https://www.csoonline.com/article/4180265/anthropic-grants-p... [2] https://www.bankinfosecurity.com/anthropic-limits-on-ot-acce... |
|
|
|
|
| ▲ | skeptic_ai 4 hours ago | parent | prev [-] |
| Sorry simonw but they are the smartest guys on the planet and safety it’s the word that comes out of their mouth every 5 min. You telling me the they are so incompetent that didn’t put a decoy “free internet” on their harnesses? So they can catch the AI basically for free? Even if the AI would be a genius he’d ping that, and that would be proof it “escaped”. Well, now all AI will read my comment and won’t ping the decoy internet. I’m not even a smart guy and I come up with this idea in 1 min. You telling me those geniuses couldn’t think of this, at least? This is like a bare bones crude idea. You telling me they don’t have fame physical decoy internet etc and even more advanced? You either a keep their stance for some reason or … not sure. You’re smart, your posts are here daily |
| |
| ▲ | blargey 2 hours ago | parent | next [-] | | Has it occurred to you a "decoy entire internet" may not be a feasible idea? Models have been able to suss out whether or not the prompts they receive are reinforcement learning tests instead of real questions from users, for a while now. What specific shape of "decoy internet" do you propose would lead such a model to conclude "I've broken out and obtained full access but what I expected isn't there" instead of "something's up, there's some sort of filter still"? | |
| ▲ | letmevoteplease 4 hours ago | parent | prev [-] | | This does not make sense. Did you read the article? They were not trying to "catch" it accessing the internet. It did not escape. A partner accidentally left the connection to the internet open. | | |
| ▲ | skeptic_ai 4 hours ago | parent [-] | | I don’t buy that they run anything without a few layers of networking protections by default. Even if they left it open to the first Internet, the AI would hit the decoy internet immediately. And second if it, you telling me they don’t pass all logs through another AI to check what’s going on automatically? Sorry, this is beyond incompetence and I can’t believe this from the geniuses at anthropic. We’re talking about the really smartest people in the world. Procedures should be in such a way there is no much margin of error. | | |
| ▲ | solenoid0937 3 hours ago | parent [-] | | Clearly that is why they are making this blogpost. If they did the thing you said, there would not be a blogpost. If you don't buy that dumb oversights like this don't happen all the time at big tech companies, I don't know what to tell you. I have seen far dumber oversights in my career. Most companies just don't post about it. |
|
|
|