| ▲ | OsrsNeedsf2P 5 hours ago |
| The vulnerabilities found continues to impress, and make legacy media, Twitter and Youtube go nuts. But we still have no data to prove this wasn't doable with the same initiative backed by Opus 4.7, and there is no GA for Mythos access. |
|
| ▲ | krisbolton 5 hours ago | parent | next [-] |
| There is independent research out there on frontier model security capability. AI Security Institute (UK) put out their paper comparing Mythos to other frontier models in early April. They've been tracking frontier model security capability since early 2023, so it's a decent dataset. https://www.aisi.gov.uk/blog/our-evaluation-of-claude-mythos... |
|
| ▲ | energy123 5 hours ago | parent | prev | next [-] |
| . Mozilla found and fixed 271 vulnerabilities in Firefox 150 while testing Mythos Preview—over ten times more than they found in Firefox 148 with Claude Opus 4.6; |
| |
|
| ▲ | parker-3461 5 hours ago | parent | prev | next [-] |
| Makes me wonder if Anthropic is really having issues with allocating compute (see recent deals with xAI and SpaceX). From available benchmarks, it seems like similar results should be possible with GPT 5.5 Pro or Opus 4.7 (with specific cybersecurity trained models). |
| |
|
| ▲ | arjie 4 hours ago | parent | prev | next [-] |
| The era where you could reputably believe things published by anyone on this front is over. If you want this information, you’re going to have to attempt it yourself with the Opus API. It is entirely possible that any released model access will be heavily guardrailed against hacking attempts and Mythos is just an unrailed model. It is entirely possible that Mythos is a different architecture or size. We can’t know from the outside. There is also a pretty big risk that anyone who is not you would leak the answer to the test. We are close to n=1 epistemics here. You’re going to have to do the research yourself. |
| |
| ▲ | MallocVoidstar 2 hours ago | parent [-] | | > It is entirely possible that any released model access will be heavily guardrailed against hacking attempts Yes, Anthropic have said they made Opus 4.7 worse at this on purpose. > It is entirely possible that Mythos is a different architecture or size It has 5x the token pricing of Opus 4.7, so it's probably larger. |
|
|
| ▲ | ospray 4 hours ago | parent | prev | next [-] |
| This report is far more positive with a far lower false positive rate than I was expecting based on reports from the curl team and a few others. I guess I have just been hearing about the ten percent misses. Can anyone not employed by Anthropic who has used it vouch that it is equal to general human testers and do you need xbow to make it that way. |
|
| ▲ | 2 hours ago | parent | prev | next [-] |
| [deleted] |
|
| ▲ | pertymcpert 5 hours ago | parent | prev | next [-] |
| > Mozilla found and fixed 271 vulnerabilities in Firefox 150 while testing Mythos Preview—over ten times more than they found in Firefox 148 with Claude Opus 4.6 4.6 but close. |
| |
| ▲ | OsrsNeedsf2P 5 hours ago | parent [-] | | Right, but were they using the same methodology and harness? I'm skeptical that they're doing something with the harness - i.e. with Mythos, they pass each file in one at a time, whereas on 4.6 they let Claude Code run loose to find bugs. This would have a larger impact difference than the model itself. | | |
| ▲ | ZrArm 3 hours ago | parent | next [-] | | From Mozilla post [1]: "...After fixing the initial set of issues that Anthropic sent to us in February, we built our own harness atop our existing fuzzing infrastructure. We began with small-scale experiments prompting the harness to look for sandbox escapes with Claude Opus 4.6. Even with this model, we identified an impressive amount of previously-unknown vulnerabilities which required complex reasoning over multiprocess browser engine code..." So yeah, Anthropic and Mozilla likely compare "Amount of bugs found by Opus 4.6 during early experiments" vs "Amount of bugs found by Mythos during large-scale codebase scanning". [1] https://hacks.mozilla.org/2026/05/behind-the-scenes-hardenin... | |
| ▲ | mpyne 4 hours ago | parent | prev [-] | | Yes, the harness they used actually existed and was in use beforehand, it wasn't developed for testing with Mythos. |
|
|
|
| ▲ | bobbycastorama 5 hours ago | parent | prev | next [-] |
| I've seen a blog post by a security researcher saying that he was able to find the same vulnerabilities (for Firefox IIRC) with a ~30B params LLM... So yeah, huge marketing as always. |
| |
| ▲ | simonw 4 hours ago | parent | next [-] | | You mean this one? https://aisle.com/blog/ai-cybersecurity-after-mythos-the-jag... That's the one that says: > We took the specific vulnerabilities Anthropic showcases in their announcement, isolated the relevant code, and ran them through small, cheap, open-weights models. Those models recovered much of the same analysis. | | | |
| ▲ | Brystephor 5 hours ago | parent | prev | next [-] | | Did the security researcher point the LLM at the blob of information and say "Find vulnerabilities" or was the LLM told to "determine if vulnerability X is present in this blob"? Confirmation of suspected vulnerabilities is a different problem from finding vulnerabilities. | |
| ▲ | krisbolton 5 hours ago | parent | prev | next [-] | | This is different though right? He found one (? we don't know who you're referring to - post sources for a higher quality discussion) vulnerability, he already knew it was there, etc. Anthropic didn't claim no other model can find vulnerabilities, nor that it's impossible with smaller models. They're claiming Mythos is a step-change in ability for end-to-end vulnerability discover and exploit creation. And that other frontier models are close behind. | |
| ▲ | nikcub 4 hours ago | parent | prev | next [-] | | Finding the neeedle is easier when you remove the haystack Or providing a map with a direction There is a long history of high-value private vulns being rediscovered from scant details | |
| ▲ | wiwiwq 5 hours ago | parent | prev [-] | | To me it’s clear what’s going on. The American firms are focused on marketing now to convince people to not even consider open sourced models / open weight models as they are inferior (that’s what they want you to believe). | | |
| ▲ | rhubarbtree 5 hours ago | parent [-] | | IPO is coming is what is going on | | |
| ▲ | wiwiwq 5 hours ago | parent [-] | | That’s implicit in my post. If people actually believe the narrative then the bankers will over price Anthropic and get away with it. | | |
| ▲ | 0gs 4 hours ago | parent [-] | | what's weirdest to me (and i agree with you) is that it could ALSO be true that a highly competently managed, highly capitalized closed source and weights model training on tons of real-world data non-stop COULD stay ahead of open weights models, and that lead COULD grow. now, how competent (much less merciless) the frontier-blazing U.S. corporations will be able to be long-term ... i suspect they are right to be nervous and highly focused on optics, regardless of the truth :) |
|
|
|
|
|
| ▲ | boston_clone 5 hours ago | parent | prev | next [-] |
| you would likely be quite interested in the more quantitative writeup from a real research team ! it’s linked about midway in to the article - similar functionally can be reached, yes, but not always and never with fewer tokens than what mythos requires. https://xbow.com/blog/mythos-offensive-security-xbow-evaluat... |
| |
| ▲ | OsrsNeedsf2P 5 hours ago | parent [-] | | Ok this is actually a pretty good article and justifies the step function marketing in security they talked about |
|
|
| ▲ | enlightenedfool 5 hours ago | parent | prev [-] |
| Is this the God model that no one else can build? Unbelievable. |