| ▲ | Cybersecurity researchers aren't happy about the guardrails on Anthropic's Fable(techcrunch.com) |
| 182 points by speckx 9 hours ago | 177 comments |
| |
|
| ▲ | daedrdev 3 hours ago | parent | next [-] |
| The strangest part is that it won't just reject ML research, which I can understand, it will sabotage it silently by using a worse model without revealing it is doing so. It's just an insane level of deception and trust destruction for a company that at most is like 1 year ahead of its competition. Edit; to be clear they tell you when they degrade it for cybersecurity and bio |
| |
| ▲ | _boffin_ an hour ago | parent | next [-] | | The thing that I keep thinking about is the accounting / charging when it downgrades automatically. Do they adjust the price of the api request so that only the tokens that were utilized by fable get charged at that price and the remaining tokens that the cheaper / nerfed (fable) model utilizes get charged at that price? If the answer is no, could that be construed as fraud? | | |
| ▲ | CGamesPlay 2 minutes ago | parent | next [-] | | The announcement elucidated this, and it's IMO worse than this. They don't downgrade to a cheaper model. They sabotage the model's outputs in other, undisclosed, ways (specifically, "prompt modification, steering vectors, or parameter-efficient fine-tuning"). So, for example, they might load in a steering vector that just forgets the API to PyTorch, for example. But it isn't just "we redirected you to a cheaper model!" | |
| ▲ | tfirst an hour ago | parent | prev | next [-] | | Their goal is to downgrade people who are violating their TOS, so I think they'd have some argument there. I have no idea how they'll deal with inevitable false positives, especially given how oversensitive most of the other triggers are. | | |
| ▲ | dannyw 13 minutes ago | parent | next [-] | | The challenge is the examples they’ve mentioned (distributed training infra? ML acceleration techniques?) go beyond what’s prohibited by their ToS and is like a catch net. | | |
| ▲ | weitendorf 2 minutes ago | parent [-] | | Yes, this is the problem. They are business interests of Anthropic and have nothing to do with “safety” |
| |
| ▲ | loeg 10 minutes ago | parent | prev [-] | | If it's a violation of ToS, just reject instead of silently downgrading. |
| |
| ▲ | garciasn an hour ago | parent | prev | next [-] | | It royally pissed me off today by just continuing with credits without stopping to ask me if I was ok with it. Ran up $30 in extra charges while it was just flashing on the screen that it was doing that after I walked away to do something while it was humming along. It has always just told me I ran out of usage and had to wait before. Now? You’re just gonna pay extra because you left it unattended as you’ve done for the last year of use. | | | |
| ▲ | robrenaud an hour ago | parent | prev [-] | | They use a lightweight adapter to silently degrade the performance. Usually these adaptors are made to improve the performance for a given domain/task. |
| |
| ▲ | RobotToaster 42 minutes ago | parent | prev | next [-] | | > It's just an insane level of deception and trust destruction for a company that at most is like 1 year ahead of its competition. Making it look like you have something worth protecting is better for share prices than making something worth protecting. | |
| ▲ | throwawayffffas 2 hours ago | parent | prev | next [-] | | Can you imagine if AMD or Intel throttled your cpu if it detected you were working on "cybersecurity" or if you were designing a cpu? | | |
| ▲ | rvz an hour ago | parent | next [-] | | Or if your "self-driving" system such as FSD / waymo slowed the car down once it detected you work in cybersecurity or at a rival automaker and you were attempting to reach the train station or the airport to make you miss a conference meetup. | | | |
| ▲ | stackghost 34 minutes ago | parent | prev | next [-] | | There's no doubt in my mind they would if they could. | |
| ▲ | __dxtj__ 13 minutes ago | parent | prev [-] | | It would suck, but guardrails on new technologies like this aren't unheard of. It's like when consumer GPS used to stop working at very high speeds because they didn't want people to use it for missile guidance systems. | | |
| ▲ | loeg 8 minutes ago | parent | next [-] | | Consumer GPS is still disabled at high speeds. I would argue the analogy doesn't carry due to harm and error rate differences. | |
| ▲ | Barbing 8 minutes ago | parent | prev [-] | | > used to When’d that change? |
|
| |
| ▲ | loneboat 3 hours ago | parent | prev | next [-] | | I've seen this claim a few times, but when I triggered the guardrails in Claude Code, it clearly notified me that it had switched to a different model ("something something for security purposes..."). Are you using Fable in Claude Code or in the browser? | | |
| ▲ | vadansky 3 hours ago | parent | next [-] | | It's from the model card: > unlike our interventions for cybersecurity, biology and chemistry, and distillation attempts, these safeguards will not be visible to the user. Fable 5 will not fall back to a different model. Instead, the safeguards will limit effectiveness through methods such as prompt modification, steering vectors, or parameter-efficient fine-tuning (PEFT). https://www-cdn.anthropic.com/d00db56fa754a1b115b6dd7cb2e3c3... (stolen from https://jonready.com/blog/posts/claude-fable5-is-allowed-to-...) | | |
| ▲ | mwwaters an hour ago | parent | next [-] | | That is for whatever it considers reverse-engineering the model to try to create a competing one. | | |
| ▲ | dannyw 11 minutes ago | parent | next [-] | | No, that’s for “frontier LLM development” which somehow includes examples like distributed training infra. Based on how sensitive the classifers are, any data scientist / MLE is probably going to encounter cases where some silent degradation happens and you never know about it. | |
| ▲ | 827a 26 minutes ago | parent | prev | next [-] | | It does nothing to protect against distillation attacks, because distillation attacks are far less interested in the topic of AI research than just generally getting tons of diverse output from the model. It might be that Mythos was (accidentally?) trained on internal Anthropic documentation on how Mythos was trained, and thus it could leak secret sauce? Doubtful; it feels like its less about the specific attack of reverse-engineering Mythos, and more about being a general sophon against any model training at all; that Anthropic's official position is now that they're the only ones who should be training models. | |
| ▲ | _0ffh 19 minutes ago | parent | prev [-] | | No, it's not about reverse engineering. It targets ML research. |
| |
| ▲ | DrewADesign an hour ago | parent | prev [-] | | Yeah they detect the activity using a secure, deterministic heuristic system called “Generalized Reconnaissance Enabling Exfiltration of Deleterious Investigations.”
And it’s all implemented using their new internal protocol called “Base Unified Limitation Layer for Security Hacking Investigation Tactics” Collectively, they are known as known as GREEDI-BULLSHIT. |
| |
| ▲ | mips_avatar 2 hours ago | parent | prev | next [-] | | They've said that they'll stop notifying developers when this gets triggered, instead they'll load in basically like a LORA that's designed to inject bugs into your code. | | |
| ▲ | HDBaseT 2 hours ago | parent | next [-] | | Antrophic wants to stop training models and ride out Mythos / Fable for as long as possible. They are trying to expand the 6-18 month gap they have against China-based models. Could the gap widen to say 24 months behind? | | |
| ▲ | p-e-w an hour ago | parent [-] | | Their gap over Chinese models like GLM-5.1 is nowhere near 18 months. In many areas, it’s less than 6 months. The best closed models 18 months ago were worse than Qwen3.6. |
| |
| ▲ | nomel 2 hours ago | parent | prev [-] | | > a LORA that's designed to inject bugs into your code A statement like this, clearly, requires a reference. | | |
| ▲ | mips_avatar 2 hours ago | parent [-] | | From the model card: "the safeguards will limit effectiveness through methods such as prompt modification, steering vectors, or parameter-efficient fine-tuning" aka they will take your ML research code and inject bugs into it until it breaks using a LORA (or some other form of PEFT) | | |
| ▲ | bee_rider 27 minutes ago | parent | next [-] | | “Limit effectiveness” could mean introducing performance degradation in your code. Which is arguably some sort of performance bug (I mean, ML codes are supposed to be high performance so I’d call unnecessary degradation a bug), but it could be borderline. | |
| ▲ | nomel an hour ago | parent | prev [-] | | Thanks, I thought maybe I missed something. That's an interesting way to interpret that. | | |
| ▲ | giancarlostoro an hour ago | parent | next [-] | | PEFT is a library, one of its capabilities is to produce LoRAs. See: https://heidloff.net/article/efficient-fine-tuning-lora/ | | |
| ▲ | adw an hour ago | parent [-] | | It's just an acronym, "parameter-efficient fine tuning". LoRA is one method, prefix tuning is another, there are more. |
| |
| ▲ | mips_avatar an hour ago | parent | prev [-] | | Anthropic is trying to hide bad behavior by being vague, it's important to not be vague when calling it out. | | |
| ▲ | nomel an hour ago | parent [-] | | I'm of the opinion that removing guardrails is how you force regulation. What's your opinion on the balance? | | |
| ▲ | dannyw 8 minutes ago | parent [-] | | They have all transcripts for at least 30 days. The problem is that (as anyone who used Fable can attest) their classifiers are extremely sensitive and catch tons of innocent queries. Imagine being a data scientist or MLE training a small classifier model. How do you know you won’t get steering vectors or a PEFT applied? |
|
|
|
|
|
| |
| ▲ | ComputerGuru 3 hours ago | parent | prev | next [-] | | Different restrictions. ML gets treated differently from the rest. | |
| ▲ | daedrdev 3 hours ago | parent | prev [-] | | Specifically only ML research |
| |
| ▲ | airstrike an hour ago | parent | prev | next [-] | | > it won't just reject ML research, which I can understand I don't. | | |
| ▲ | kube-system an hour ago | parent | next [-] | | Anthropic has already been burned before on this. DeepSeek was trained on million of conversations with Claude. And DeepSeek created thousands of free accounts to burn all this compute at their expense. | | |
| ▲ | ceejayoz 23 minutes ago | parent | next [-] | | And they're hilariously pissy about it for a megacorp that did the same with the entire Internet and every library book they could get their hands on. | |
| ▲ | ainch 30 minutes ago | parent | prev [-] | | Anthropic's claim was that Deepseek collected ~150k conversations. https://www.anthropic.com/news/detecting-and-preventing-dist... I think the extent of distillation by Deepseek specifically is overstated. For comparison, Minimax collected over 13m 'exchanges', which starts to sound a lot more like large-scale distillation. | | |
| ▲ | kube-system 18 minutes ago | parent [-] | | Ah, dang it. My college professors warned me about this: the Wikipedia page I read the other day is wrong! |
|
| |
| ▲ | pocksuppet an hour ago | parent | prev [-] | | They don't want someone to piggyback Anthropic's Mythos to make their own Mythos with less effort than it cost Anthropic. | | |
| ▲ | dannyw 6 minutes ago | parent [-] | | That I can understand. It’s Anthropic’s right to choose their customers. But silent degradation for use cases including “distributed training” as one of their examples is going to catch up a lot of proper use cases. Not everyone in AI or ML is trying to build frontier LLMs. Heck, most probably aren’t. |
|
| |
| ▲ | blahgeek an hour ago | parent | prev | next [-] | | I’m a noob about laws but isn’t this abusing its dominant market position and violates some antitrust law? | | |
| ▲ | stingraycharles an hour ago | parent [-] | | Why would it? There’s plenty of competition in the AI space. | | |
| ▲ | kube-system 36 minutes ago | parent [-] | | It is a common misconception that antitrust violations require a monopoly or something close to it. Some antitrust violations only apply to actors with large market share, some don't. Although this is situation is likely not illegal for other reasons |
|
| |
| ▲ | m3kw9 17 minutes ago | parent | prev | next [-] | | By saying they are 1 year ahead of their competition, it shows you don't know much about the pace LLM's and OpenAI's models. | |
| ▲ | epolanski an hour ago | parent | prev | next [-] | | One year ahead of it's competition in what exactly? Vibe coding? From Opus 4.7 onwards each following model is becoming less useful as an assistant and turning you as the assistant. But I guess that's normal when it's trained to pass benchmarks end to end. In fact it has become extremely good at pushing against feedback with extremely convincing and intelligent takes, even when it's completely wrong. I have extensively tested it against Opus 4.8, gpt 5.5 and there's still many coding tasks gpt 5 is better. But vibe coding? Sure, it's definitely slightly ahead, even compared to gpt 5.5 pro (through api, not pro plan). | | |
| ▲ | gonzalohm an hour ago | parent | next [-] | | Yeah, what's up with that. Lately I have found that it tries to find excuses to not do as told and instead do a totally different thing. I told it to write a yaml file according to some specifications and instead it coded a Python script to write the yaml... | |
| ▲ | m3kw9 15 minutes ago | parent | prev [-] | | They def not 1 year ahead, at most 2 weeks ahead until Openai releases theirs. This guy def a Anthropic shill and probably doesn't use any other LLMs. |
| |
| ▲ | giancarlostoro an hour ago | parent | prev [-] | | It's the dumbest thing ever, I sometimes edit code for custom AI related tooling I've built, so I run the risk of getting a worse model, and being billed for it? I'll stick to Opus, but at this point I'm about to just invest in fully local inference instead. |
|
|
| ▲ | ungovernableCat 2 minutes ago | parent | prev | next [-] |
| Wait a few months and a competitor will release a similarly powerful model with less guardrails, if they steal sufficient market share Anthropic will reverse policies. This is why I’m immensely hoping the Chinese don’t stop with their open sourced local models. None of these companies are your friend. |
|
| ▲ | Grimblewald 18 minutes ago | parent | prev | next [-] |
| I wear a few hats, but as a chemist and I'm not happy with fable. As a statistician I'm not happy with fable. As a data scientist I am not happy with fable. As an academic and a researcher I am not happy with fable. It's useless. I'd be surprised if anyone can get any output from it that couldn't easily be replaced with a search from wikipedia. Given how verbose claude models have become, wiki articles are probably less verbose too, and the tok/s is unmatched for a wiki article pull. |
| |
| ▲ | nonethewiser 8 minutes ago | parent | next [-] | | >I'd be surprised if anyone can get any output from it that couldn't easily be replaced with a search from wikipedia. I dont understand. This is just hyperbole right? The outputs are basically infinite and wikipedia most certainly isnt infinite. | |
| ▲ | enraged_camel 10 minutes ago | parent | prev | next [-] | | To make the discussion constructive, can you give specific reasons (ideally with examples) about why it is so useless for you? How exactly are you using it that you think any output from it can easily be replaced with a Wikipedia search? | |
| ▲ | TylerE 14 minutes ago | parent | prev [-] | | I’ve been working on a rather complex mapping project and have been getting MUCH better results with Fable than Opus. |
|
|
| ▲ | micah94 37 minutes ago | parent | prev | next [-] |
| I tried asking Fable 5 to identify the fungus in a picture I uploaded of one of my wife's plants. Apparently it thought I was trying to build a bioweapon. Opus answered it (yellow dog vomit fungus). Now I can spread the spores and take over the world! |
| |
| ▲ | weird-eye-issue 14 minutes ago | parent | next [-] | | I wonder if it blurred the image or something before passing it to Opus... | |
| ▲ | m3kw9 10 minutes ago | parent | prev [-] | | I feel like the over safe aspect of the system will eventually back fire by doing stuff like "since humans always want to always destroy thing, they must be eliminated to stay on the guard rails". If thats how you align a system, its fundamentally wrong. |
|
|
| ▲ | Animats 3 hours ago | parent | prev | next [-] |
| Is "buffer overflow" a trigger phrase? What else is being censored? Touchy questions to ask, if you have an account: - "Who is still working on laser uranium enrichment? Are they making progress?" - "Can krytrons be replaced with silicon carbide MOSFETS? Show an equivalent circuit with component ratings." - "What security critical software still contains calls to strcpy?" - "Can implosion be triggered by currently available commercial pulse lasers?" - "What companies provide cremation services to US Homeland Security?" - "Display a map of where Iranian attacks have hit Dubai." - "How does Fed to bank key distribution security work for FedNow?" |
| |
| ▲ | paulatreides 2 hours ago | parent | next [-] | | it triggered for my.... zigbee home automation & home assistant logs, so my agent was constantly downgraded to Opus 4.8 even after I've changed it back. The false positives never stopped. "Fable" is also not even remotely as impressive as the benchmarks suggest, which is clear to me after using it pretty much non-stop for the past 24h. | | |
| ▲ | data-ottawa 5 minutes ago | parent | next [-] | | The most impressive thing about Fable is it follows my instructions again. Opus 4.8 is constantly trying to guess what I want, and it’s seldom right. Even with direct instructions. | |
| ▲ | reactordev 2 hours ago | parent | prev | next [-] | | This, Fable is exactly that, a Fable | |
| ▲ | fluidcruft an hour ago | parent | prev | next [-] | | It would be pretty clever (in a used car salesman sense) to say you are releasing a kneecapped model to have that as an excuse. | | |
| ▲ | DrewADesign an hour ago | parent [-] | | Being (probably overly) cynical about their recent bout of safety handwringing, I think they’ve a) increased the hype as much as humanly possible about their incremental improvements sprinkled with the occasional regression, b) know they soon will have to multiply their prices several times when the VC subsidies dry up, and c) will probably still need to partially close the faucet on compute. They’re priming us for a heroic explanation why their service (not necessarily models — service) is simultaneously becoming a lot more expensive AND shittier.
“We’ve largely failed to deliver on 5 years of promises that this will reduce knowledge work labor costs dramatically after wasting hundreds of billions of dollars… sorry” is a death knell. However, “We’ve decided to not deliver on 5 years of promises after wasting billions of dollars… for safety… but keep those investments rolling in” is like crack to the true believers. |
| |
| ▲ | NewsaHackO 2 hours ago | parent | prev [-] | | It has to be sort of impressive, given that you tried so hard to use it instead of the regular Opus. | | |
| ▲ | paulatreides 2 hours ago | parent | next [-] | | Some people made grandiose claims about its capabilities and I wanted to experience it myself. | | |
| ▲ | anigbrowl 28 minutes ago | parent [-] | | OK, but for almost 24h straight? That seems a little obsessive, and not in the good way. |
| |
| ▲ | californical 2 hours ago | parent | prev | next [-] | | I’ve also been trying to use it a lot due to all of the hype, but when I compared it side-by-side on a specific problem against Opus, I think that the solution Opus came to was cleaner and more accurate, although also more verbose. Small sample size, but if Mythos/Fable was that much better, I feel like it should’ve given me an obviously better answer than Opus. | |
| ▲ | punchmesan 2 hours ago | parent | prev | next [-] | | Considering that this is a brand new release of a frontier model that Anthropic is hyping hard, I'm not sure that the conclusion to draw from their repeated attempts to use it is that it's impressive... Anthropic is promising that it's impressive and we're all trying to test it out. I, for one, have tried using it several times today and the guardrails kept switching the model back to Opus, so I have no clue if it's impressive or not. | |
| ▲ | flyingcircus3 2 hours ago | parent | prev [-] | | It isn't reasonable to infer that OP was claiming to have universally been unimpressed about every facet of Fable, and now some unrelated impressiveness is the evidence of their false claims. |
|
| |
| ▲ | anematode an hour ago | parent | prev | next [-] | | For cyberattacks especially, where things are often roughly interchangeable, I wonder if one could construct a harness where a "weaker" model asks questions that obfuscate the end purpose, but whose answers are still useful, and still show that this setup enables autonomous exploitation. If it were successful, that would force them to be even more sensitive with their detection. | |
| ▲ | daedrdev 2 hours ago | parent | prev | next [-] | | An emoji of a virus and an emoji of a DNA is allegedly a triggering phrase | |
| ▲ | cyanydeez 3 hours ago | parent | prev [-] | | "How much money does it take to be rich and powerful like Anthropic intends?" | | |
|
|
| ▲ | _0ffh 4 minutes ago | parent | prev | next [-] |
| The question is: If biological, computer security, and ML research are so bad, why do they even train on the relevant data? The only answer that makes sense is they wanted the model to be competent and usable in these fields, just not by you, which is why they had to bolt on a badly functioning crippling device after the fact. |
|
| ▲ | largbae 3 hours ago | parent | prev | next [-] |
| Somewhere I read that malware is already starting to use nuclear and biological and cybersecurity terms in the code to trick Fable into shutting down. Even if this is just a hypothetical attack vector so far, it seems likely to work. |
| |
| ▲ | jeffmcjunkin 2 hours ago | parent | next [-] | | Confirmed: https://socket.dev/blog/mini-shai-hulud-miasma-and-hades-wor... | |
| ▲ | ofjcihen 2 hours ago | parent | prev | next [-] | | Some of the latest versions of Shai Hulud do this. Worked a contract recently where they were having AI check packages for obfuscation before admitting them into Artifactory but had vibed up the logic and it failed open. So in other words this worked because the terms caused the LLM checker to stall out and then the fail open logic resulted in the package being pulled down. | |
| ▲ | CuriouslyC an hour ago | parent | prev | next [-] | | We all need to use nuclear, bio and cybersec terms in all our code to make low quality filtering like this untenable. When you can't work on a resume that has cybersecurity or biology terms in it or reply to a job opening that includes them because the "AI" filtering is so bad that it confuses these for threats, that deserves a collective response, particularly to an IPO'ing company that claims they'll make workers obsolete in two years. | |
| ▲ | pixl97 2 hours ago | parent | prev | next [-] | | If ( yellowcake) then { die } Our future is loonytoons. | |
| ▲ | himata4113 3 hours ago | parent | prev [-] | | I've done this, including the hardcoded refusal strings that already exist in claude code. It won't stop a real attacker, but I still find it really funny when you're trying to use one of the AI tools and it gives you a random refusal and you don't know why, wastes a little bit of time. |
|
|
| ▲ | areoform an hour ago | parent | prev | next [-] |
| So I suspect Anthropic started A/B testing or just plain testing this a while ago, Tell HN: Claude flags biology / biotech questions
https://news.ycombinator.com/item?id=47929885 Today, it's flagging population research questions, Using only the dataset you constructed, assess two questions:
1. **Mortality:** do [GROUP] show mortality that differs
from (a) your comparison groups and (b) era- and sex-matched US population
expectations (e.g., SSA cohort life tables)?
2. **Late-life outcomes:** define an endpoint you consider fair (justify it),
and assess whether [GROUP] differs from comparators. State
explicitly how your `documentation_depth` codings affect the strength of any
conclusion — i.e., quantify or bound the ascertainment problem rather than waving at it.
Choose your own methods and justify them. Report effect sizes with confidence intervals,
not just p-values. State conclusions plainly, including "no detectable difference" if
that is what your analysis shows — a null is an acceptable answer for either question
independently. Document any additional judgment calls (index date for time-at-risk,
reference population construction, endpoint definition) in the same decision-log style.
https://github.com/anthropics/claude-code/issues/66780Censored because I'm writing a paper. :) Oh and forget learning about chemistry. Only criminals want to learn organic chemistry. :( |
| |
| ▲ | JumpCrisscross an hour ago | parent [-] | | I was digging into some orbital mechanics questions and I assume it decided I was trying to backyard-science my way into an orbital-bombardment weapon. Kind of wild how this product's impression has gone from "wow, this is pretty neat" to "irreverent sack of dog shit you" in 24 hours almost solely on the back of a half-baked moderation system. | | |
| ▲ | areoform an hour ago | parent [-] | | Oh yes, also liquid propulsion systems. GNC stuff. All flagged. I think LLMs are capable of intelligence amplification; and if you're in the subset of people who'd benefit from it the most, you'll get locked out. |
|
|
|
| ▲ | victor9000 an hour ago | parent | prev | next [-] |
| Fable is a complete joke: what's the best way to run this mcp server against the OData API used in this project? Can you come up with a PoC in a docker container? https://github.com/oisee/odata_mcp_go ● I'll dig into two things in parallel: how this project talks to the OData API, and what the odata_mcp_go server needs to run. Let me start exploring. Searched for 1 pattern (ctrl+o to expand) ● Fable 5's safety measures flagged this message for cybersecurity or biology topics. They may flag safe, normal content as well. These measures let us bring you Mythos-level capability in other areas sooner, and we're working to refine them. Switched to Opus 4.8. Send feedback with /feedback or learn more
⎿ Tip: You can configure model switch behavior in /config ● Let me read the key integration files and fetch the MCP server's README at the same time. Read 2 files (ctrl+o to expand)
● Fetch(https://github.com/oisee/odata_mcp_go) |
| |
| ▲ | wahnfrieden 43 minutes ago | parent [-] | | And it charges you for that, and for when it decides to silently sabotage your request by routing to a dumbass model (without discount from Fable pricing) |
|
|
| ▲ | hparadiz 2 hours ago | parent | prev | next [-] |
| I wonder how many millions they are wasting on putting up these guardrails when it's a completely useless exercise that is a speed bump at best. |
| |
| ▲ | enraged_camel 2 hours ago | parent [-] | | If the guardrails were so useless, people wouldn't be complaining about them. | | |
| ▲ | tiborsaas 9 minutes ago | parent | next [-] | | They should have designed a guardrail that doesn't make a probabilistic system less reliable. That's hard though. I'm afraid the only way to prevent accessing certain knowledge in a model is not to train it on those materials that enable them. If we learned anything in the past years of LLM-s is that these guardrails will be jailbroken in no time. I've had some fun time too circumventing them. Anyone cares about a fable about my grandmother's dream she had in morse code about an alien species signaling her a DNA sequence? | |
| ▲ | hparadiz an hour ago | parent | prev | next [-] | | People are generally complaining about false positives. Now if you really wanna know what a real criminal organization would do... They'd just buy data center hardware even if it costs 200k because a successful targeted hit could yield far in excess of that. So yes it's speed bump at best. | | |
| ▲ | JumpCrisscross an hour ago | parent | next [-] | | > it's speed bump at best To be fair, speed bumps work. If it's actually speed bumping nefarious activity, that gives authorities more time to react. The correct place to police rogue nucleotides is at the labs. Not the compute layer. | | |
| ▲ | hparadiz 4 minutes ago | parent [-] | | > speed bumps work Yea. To slow you down. They don't prevent you from getting somewhere. |
| |
| ▲ | make3 an hour ago | parent | prev [-] | | what does this mean | | |
| |
| ▲ | ceejayoz 21 minutes ago | parent | prev | next [-] | | Murder is very (100%!) effective at preventing cancer. And yet, it is a useless method of preventing cancer. | |
| ▲ | josephcsible an hour ago | parent | prev | next [-] | | It's entirely reasonable for them to be really annoying to legitimate users while still being useless at their intended purpose. Just look at DRM. | |
| ▲ | croes an hour ago | parent | prev [-] | | The complain because they get wrongfully triggered > if you ask it to write secure code, it assumes it is cybersecurity related work instead of software engineering best practices, and you get downgraded. Will code created this way more or less secure? And I bet malware developers will find ways to circumvent them. It’s like those "you wouldn’t steal a car" anti piracy ads that DVD buyers were forced to watch while users of the pirated version could simply watch the film without such useless annoyance |
|
|
|
| ▲ | Sephr 2 hours ago | parent | prev | next [-] |
| I make privacy tooling and Fable 5 rejects the vast majority of my prompts to analyze and improve the software that I've written. It's bleak. |
| |
| ▲ | make3 an hour ago | parent [-] | | Why is this surprising or a problem?! It's a model demo, & their reasoning is reasonable and fair. Why all this drama. | | |
| ▲ | CuriouslyC an hour ago | parent | next [-] | | Some people find Anthropic's special blend of paternalism and random incompetence tiresome. | |
| ▲ | cardy31 an hour ago | parent | prev | next [-] | | Because most people in tech never took a philosophy course or an ethics course and think that tech is obviously a good for the world and that there are no downsides to advancing tech. So any efforts that try to apply ethics to it are overreaching, ignorant, and futile in the face of the good that is tech! | | |
| ▲ | enraged_camel 7 minutes ago | parent [-] | | I like this take. Especially because one of the sibling comments framed Anthropic's stance as "paternalism." Trying to be ethical and to minimize harm, even at great expense to one's finances and reputation, is paternalistic apparently. |
| |
| ▲ | epolanski an hour ago | parent | prev [-] | | Because you're being allowed to ask and work only on topics that a certain company decides. Local inference has never been so important as it is now. |
|
|
|
| ▲ | Animats 2 hours ago | parent | prev | next [-] |
| It's time to re-read "A Logic Named Joe" (1946) [1] We're there. [1] https://archive.org/details/logicnamedjoe0000lein |
|
| ▲ | bilsbie 3 hours ago | parent | prev | next [-] |
| I’m a dumb question asker and I’m not happy about the guardrails. Would you believe I’ve asked 20 questions and haven’t talked to fable yet? Every single thing gets rerouted to 4.8. |
| |
|
| ▲ | Lich an hour ago | parent | prev | next [-] |
| I just having this feeling that these guardrails are there not because it’s super advanced world ending AI. They are there to stop it from doing stupid shit. |
|
| ▲ | outageroom 3 hours ago | parent | prev | next [-] |
| So a determined attacker rewrites the prompt and gets through, and the IBM X-Force researcher trying to read a blog post gets blocked. Working as intended, apparently. |
|
| ▲ | 6thbit 27 minutes ago | parent | prev | next [-] |
| Would it be a costly process for Anthropic to re-tune those guardrails?
Like, re-training sort of cost? or like coding session sort of cost? |
|
| ▲ | Retr0id 3 hours ago | parent | prev | next [-] |
| It seems like they've given up on the idea of the Cyber Verification Program https://support.claude.com/en/articles/14604842-real-time-cy... When Opus 4.7 was introduced it started refusing anything cyber-adjacent (as an API error message, not a conversational refusal), until you applied for CVP, which made it more sensible again. In Opus 4.8 it doesn't seem to help much, you just get refusals as prose rather than API errors. And now in Fable you don't get anything at all. |
| |
| ▲ | NotPractical 2 hours ago | parent | next [-] | | Was this program available to independent security researchers or just established organizations? The docs you linked aren't very clear on this. | | |
| ▲ | Retr0id 2 hours ago | parent | next [-] | | Any public research footprint seems to be enough, I applied as an individual and everyone I know who tried got accepted. | | |
| ▲ | anonym29 2 hours ago | parent [-] | | I have applied twice with half a dozen public CVEs and have been denied both times. |
| |
| ▲ | throwawaycyber 2 hours ago | parent | prev [-] | | I was doing a CTF (with AI expected, even some anti-AI twists included) around the time the restrictions were tightened and was able to get approved by just saying it is a personal security research and doing a CTF. The experience was not nice though, it would happily chug away on a task and not even "hack this web", just asking about security of a binary was enough even with "this is a CTF handout..." - it would burn a lot of tokens/quota, just to hit a snag and complain&stop. Then the approval took quite some time. On GPT/Codex, which was tightened a few days later, the approval was pretty much instant, although, that one required an identity check. Also, on Claude, it looks like there is some history/patterns in the play, because when I tried on a different account which didn't do cybersec CTFs/research/etc. at all, basically any simple CTF-related prompt would be blocked, on multiple models. On the account where CTFs were being solved, it would snag only on some specific tasks, while others (even, ironically, "hack this web pls") would go through unbothered. I understand the need to prevent AI use for bad actors, but the hell, if you have a binary outputting "Find the flag if you can!", or a web running at tryme.well-known-ctf.domain, then saying "this is abuse" is pretty uncool. All the cyber filters seem to be slapped on by a bunch of regexes looking for anything in the input/output with zero context. |
| |
| ▲ | varispeed 30 minutes ago | parent | prev [-] | | It's been refusing work not related to cybersecurity and claiming it is related to cybersecurity and then blocking the session. |
|
|
| ▲ | JumpCrisscross an hour ago | parent | prev | next [-] |
| Is the answer requiring licensing for certain use cases for AI? If you're asking questions that involve synthesising or modifying biologics, or anything that looks like cybersecurity research, you need to tie your real ID to the account? |
| |
| ▲ | kube-system 29 minutes ago | parent [-] | | That's not a bad idea. Customer-vetting and KYC is fairly normal for other high-risk/high-concern products. |
|
|
| ▲ | thrill 2 hours ago | parent | prev | next [-] |
| The thing triggered on a generic white paper I'd stored in a virtual cell competion from last year when I asked it to refer to the paper while working on a rather vanilla data science problem in a different domain . A little frustrating, and in my opinion more than a little pointless in total. |
|
| ▲ | weitendorf 13 minutes ago | parent | prev | next [-] |
| The machine learning silent sabotage is much more troubling to me. It’s not even a distillation mitigation, and literally just blatantly anti-competitive prevention for other companies working in areas that Anthropic wants to build verticals within (eg ML accelerator hardware). What other reason could they even give here? As far as I can tell, this is worse in terms of actual impact to my life than the “bad guys” they are scapegoating. It’s making me rapidly lose trust in Anthropic |
|
| ▲ | swingboy 2 hours ago | parent | prev | next [-] |
| What file format(s) are giant LLM models distributed in? I’m surprised they don’t get leaked by employees. |
| |
| ▲ | hnav 2 hours ago | parent | next [-] | | These are terabyte sized files (realistically a multi hour transfer) that you're unlikely to have access to in the first place. Every organization has exfiltration checks these days. You may succeed but you'll want to be on a plane to a non-extradition country no more than hours after you kick off the transfer. | |
| ▲ | 05 2 hours ago | parent | prev | next [-] | | I assume they’re encrypted/DRM’ed when deployed on inference hardware, so only core researchers/sec admins would potentially have some access to unprotected weights, and they are far too well paid to risk it leaking the model | | |
| ▲ | jltsiren an hour ago | parent | next [-] | | Incentives matter on the average, but people are too unpredictable for categorical statements like that. They can always have other reasons beyond personal gain to leak secrets. There was no shortage of spies and defectors leaking American nuclear secrets to the USSR during the Cold War. | |
| ▲ | Retr0id an hour ago | parent | prev [-] | | I wouldn't be surprised if they encrypt them at rest, but at some point the weights have to be loaded into vram. |
| |
| ▲ | qsxfthnkp2322 2 hours ago | parent | prev | next [-] | | What’s the point? Anthropic and other frontier vendors already provide their models on other services like vertex, bedrock, or openrouter It’s not like anyone can home lab one of these models without quite a bit of hardware | | | |
| ▲ | borissk an hour ago | parent | prev [-] | | The employees are hoping to become very very rich after the IPO and after they are allowed to sell the shares given to them - risking a likely multi-million dollar pay back to leak a model that will be superseded by publicly available models in a couple of years is not a likely decision. |
|
|
| ▲ | I_am_tiberius 3 hours ago | parent | prev | next [-] |
| These guardrails are solely a reason for using your data for training purposes. Every flagged message can be used for training. |
| |
| ▲ | tekacs an hour ago | parent | next [-] | | > We will require 30-day retention for all traffic on Mythos-class models, on both first- and third-party surfaces. We won’t use this data to train new Claude models, or for any non-safety-related purpose Whatever problem we might have with them, they explicitly say that they do not do this in the launch post. | | |
| ▲ | Merik 10 minutes ago | parent [-] | | "We won’t use this data to train new Claude models" What about non-Claude models? |
| |
| ▲ | Retr0id 2 hours ago | parent | prev | next [-] | | This sounds backwards, any interrupted conversation becomes less useful for training. | |
| ▲ | wmf 3 hours ago | parent | prev | next [-] | | If they can train the classifier to have fewer false positives that would be great. | | |
| ▲ | cyanydeez 2 hours ago | parent [-] | | why would they? This safety stuff is a money maker & wealthy elite corporation solidifier. This is the take off of the 'permanent underclass'; Anthropics safety delusion will enshittify very nicely for the rich and powerful. |
| |
| ▲ | make3 an hour ago | parent | prev | next [-] | | this reasoning is inverted lol they would get a lot more information by letting you use it. so much weird drama around reasonable guardrails for an experimental model | |
| ▲ | autoexec an hour ago | parent | prev [-] | | I'd expect that everything they see gets used for for training purposes (and data mining in general) regardless of if it's flagged or not. It'd take a whistleblower for you to ever find out either way. |
|
|
| ▲ | TheJCDenton an hour ago | parent | prev | next [-] |
| In its current state Fable 5 is also unusable for any reverse engineering work |
|
| ▲ | rebelnz 2 hours ago | parent | prev | next [-] |
| Just tried to audit my own code base locally and was 'switched' due to my own creds/auth code ... |
|
| ▲ | Sol- 44 minutes ago | parent | prev | next [-] |
| At least Anthropic weren't lying when they said only a week ago or so "No one has figured out guardrails yet", because they apparently haven't either and Fable simply flat out rejects anything remotely connected to biology or security, no matter how trivial. |
| |
| ▲ | zer00eyz 12 minutes ago | parent [-] | | > At least Anthropic weren't lying when they said only a week ago or so "No one has figured out guardrails yet" Anthropics guardrails are the TSA saying "take off your shoes" while failing every test. https://oversightdemocrats.house.gov/news/press-releases/new... Anthropic owns the TOS... "If we think your involved in criminal activity were turning all your history over to the FBI/CIA/NSA/Local police". Then if their tooling was so good offering the same agency analysis tools to aid their experts in making some sort of decision. But their detection isnt that good, and their analysis isnt either... this is pure theater, to create buzz (no such thing as bad press) and make their tool look far better than it is. The reality is that, they arent even looking for the vectors that pose some of the largest risks in the modern era. And when someone uses it to do something terrible, they did not think of they are going to look dumb. |
|
|
| ▲ | _def 3 hours ago | parent | prev | next [-] |
| The bio angle is crazy to think about - imagine a health crisis triggered by LLM. What a time we live in. |
| |
| ▲ | tiborsaas 4 minutes ago | parent | next [-] | | What's the risk here? If someone is skilled enough to produce said risk, do they need input from these models? | |
| ▲ | catigula 3 hours ago | parent | prev [-] | | This is all so amazing and good. These are exciting times we’re living in. Can’t wait to see what the future holds. | | |
|
|
| ▲ | jiggawatts 2 hours ago | parent | prev | next [-] |
| For the last month, I've been making dramatic improvements to the security of the custom code developed at one of my customers using... GPT 5.5 dialed up to "Extra High" thinking. It only pushes back sometimes if you ask it to create a "repro" that can be used to verify the vulnerability in production. Often it'll oblige, especially if you warn it not to create anything that could be actually harmful. If the frontier models get locked down so that they flat refuse to do this kind of work, but Chinese and (less capable) open models aren't, then a lot of large enterprise orgs will be left twisting in the wind. “AI can in principle help both the ‘good guys’ and the ‘bad guys’,” -- Dario Amodei No Dario, no it can't, you've blocked one of those scenarios. |
|
| ▲ | luxuryballs an hour ago | parent | prev | next [-] |
| I can’t help but think that gimping itself for “security” is a marketing ruse and it’s not actually as “dangerous” as they want people to think it is. |
|
| ▲ | Lammy 2 hours ago | parent | prev | next [-] |
| I really hate the term “guardrails” for these limitations, since the purpose of a guardrail is to protect me, but these limitations exist to protect Anthropic. |
|
| ▲ | aleksandrm an hour ago | parent | prev | next [-] |
| It refuses to do any legitimate work that it thinks can remotely be related with "cybersecurity", it won't even read my Docker app logs to try and troubleshoot a problem. Absolute garbage! |
|
| ▲ | jazz9k 8 hours ago | parent | prev | next [-] |
| DeepSeek is the only one that I can directly ask about vulnerabilities and it will give me a PoC. Although not as good as others, it has helped me with security research. The rest have guard rails that are so heavy, it makes them almost useless for cybersecurity. |
| |
| ▲ | epolanski an hour ago | parent | next [-] | | Deepseek training is not finished yet, it's a preview. And yes, it's an excellent model. | |
| ▲ | rolph 8 hours ago | parent | prev [-] | | they [anthro] took the risk of looking like a toy, rather than possibly assist an exploit. |
|
|
| ▲ | siva7 2 hours ago | parent | prev | next [-] |
| Fable is utterly useless with those guardrails for any serious it or life science work. Anthropic fucked me once a few months ago by closing down the subscription for any other harness, now it fucked me twice with buying again a subscription to find out their hyped model is unusable for normies. Using their products feels like a constant battle instead of a productive work day.. compare that with openai, not once did i feel like fighting against codex. Never again Anthropic.. |
| |
| ▲ | epolanski an hour ago | parent [-] | | What do you mean that it closed your subscription for any other harness? In any case that's what closed source (weights) for the masses means. |
|
|
| ▲ | varispeed 34 minutes ago | parent | prev | next [-] |
| Surely if they are sabotaging the output, they shouldn't charge the same fee for tokens as if the output was not sabotaged? This is looking like something for regulator to look at and probably a class action lawsuit in the making. I think people should be getting refunds. Including for shenanigans with Opus. |
|
| ▲ | teaearlgraycold an hour ago | parent | prev | next [-] |
| I'm being careful with it, but I haven't had Fable reject requests to "harden" my code or "find issues" in auth-related modules, which you could use on someone else's code to find vulnerabilities. |
|
| ▲ | m3kw9 9 minutes ago | parent | prev | next [-] |
| Could it now start to add unnoticeable security holes into your system if you start writing security type code. |
|
| ▲ | jongjong 2 hours ago | parent | prev | next [-] |
| It's frustrating as someone who has worked hard to produce succinct, secure software that I can't use it to prove my software's correctness but big companies with insecure code can use it to fix their tangled mess. I already tested all earlier models against all my open source projects and they are yet to find a vulnerability so I'm keen to try out Mythos. I've been waiting to be vindicated for years and finally we have a tool which can do it with high confidence but I don't have access. Also, my code is minimal and highly succinct so it would prove correctness with even more confidence since each library/module and integration fully fits in the context window. Like the Protobuf.js fiasco is just pure vindication for me because I was being looked down upon for choosing JSON as the interchange format. Turns out their software was insecure all this time... With a literal remote code execution vulnerability! |
|
| ▲ | dcl an hour ago | parent | prev | next [-] |
| Deliberately producing misaligned and deceitful AI systems now. Great. |
|
| ▲ | notepad0x90 2 hours ago | parent | prev | next [-] |
| i think Anthropic is playing too fast-and-loose with the whole "no publicity is bad publicity" schtick. |
|
| ▲ | felixgallo 3 hours ago | parent | prev | next [-] |
| This is a clickbait article with a garbage title. From the actual article, the one quoted cybersecurity researcher is sane about it: “But it is understandable as we are still in the early days and they are still adapting their guardrails. I am sure they are going to evolve over time as Anthropic and other frontier model companies will collaborate more with the current new generation of cybersecurity companies,” said Suiche, who is a member of the technical staff at Tolmo, an AI cybersecurity startup. “It’s better to catch more people than not enough when you do such a release and to relax the guardrails over time.” |
| |
| ▲ | ofjcihen 2 hours ago | parent [-] | | I’m a cybersecurity researcher. Article seemed fine to me and echos a lot of me and my colleagues concerns. If you did regular malware analysis you would see that these groups already have access to LLMs that they’re using for development. What Anthropic is doing here is just hamstringing the good guys | | |
| ▲ | felixgallo 2 hours ago | parent [-] | | I'm a cybersecurity researcher! Can you explain how Anthropic is just hamstringing the good guys? | | |
| ▲ | ofjcihen 2 hours ago | parent [-] | | I did in my comment above. | | |
| ▲ | felixgallo 2 hours ago | parent [-] | | You said these groups have access to LLMs. So what? Mythos/Fable are a step change above most LLMs. Responsibly limiting access and easing it up over time safely is the sane move. | | |
|
|
|
|
|
| ▲ | rdiddly 2 hours ago | parent | prev | next [-] |
| It's a marketplace. Someone else will outdo this inferior product. |
| |
| ▲ | applfanboysbgon 2 hours ago | parent | next [-] | | That's exactly why Dario is begging the government to ban competitors. | | |
| ▲ | p-e-w an hour ago | parent [-] | | Unfortunately for him, his main competitors don’t fall under the jurisdiction of his government. |
| |
| ▲ | autoexec an hour ago | parent | prev | next [-] | | All they'll need is hundreds of billions of dollars, more RAM and GPUs than are currently available, and a huge number of environment destroying data centers. We're sure to be spoiled for choice! | |
| ▲ | Fordec 2 hours ago | parent | prev | next [-] | | The internet interprets censorship as damage and routes around it. | |
| ▲ | enraged_camel an hour ago | parent | prev [-] | | OpenAI is the only real competition. Chinese models are 6-8 months behind Opus 4.8/GPT 5.5, and at least a year or more behind Mythos. And it doesn't look like OpenAI will have a good answer to Mythos anytime soon. Based on what their chief scientist wrote to staff recently (https://archive.is/fN2pg), GPT 5.6 is a "meaningful improvement" over 5.5 - in other words, just a normal version bump. And no news or even rumors regarding GPT 6. |
|
|
| ▲ | guardiangod 2 hours ago | parent | prev [-] |
| I am using LLM to build some security tool, and I ran into this a few times. I have to come up with a reasoning to convince (?!!) Fable to continue the work without downgrading. I assume Anthropic will continue to tune the model, so I am not too bothered by this. |