Remix.run Logo
PedroBatista 11 hours ago

This post and comment makes me believe "science" is the new "code" for Anthropic now that the code advantage is mostly gone and lost for OpenAI, ie. they got much better and Claude become significantly worse over these months.

rot256 11 hours ago | parent | next [-]

I write a lot of Rust and Lean, Fable 5 is in my experience better at both. Cost/performance is a different story.

black_knight 9 hours ago | parent | next [-]

Fable has become my go to in Agda as well. It just crunches hard technical tasks!

I find Fable 5 still lacking in library design. But I guess there is no accounting for taste…

airstrike 10 hours ago | parent | prev | next [-]

Yes, both of which are domains for which a verifier is readily available.

You can generalize from them to "science".

rowanG077 10 hours ago | parent | prev [-]

This is really it imo. Fable 5 is better then Sol. But Fable is just of the table for anything even remotely long running. Unless you have very deep pockets. And the difference between Fable and Sol is not world shattering if you ask me. I also find codex a ton better than claude.

prox 10 hours ago | parent [-]

So when do you use Fable? For difficult singular tasks?

rowanG077 10 hours ago | parent [-]

Yes, I use it when Sol Ultra fails to find a solution.

echelon 11 hours ago | parent | prev [-]

IMO, Codex is worse than Claude with Fable. At least at Rust.

That said, the open source models are not bad and I'm looking forward to more tools and products built on top of them. Code review, security review, etc.

Anthropic needs to change how it treats users though. I'm increasingly put off by Dario, the rug pulling, the lies, and the attempts to regulate open weights. I'm going to bail if this doesn't change. There's plenty enough that's good enough, and those things are hackable and extensible.

If Fable isn't available at subscription price via third party harnesses soon, I'm also going to bail.

ed-is-ai 7 hours ago | parent | next [-]

The big issue I have with Fable is this. From the Anthropic email announcing Fable 5.1. So basically they're giving us a Ferrari, which will point blank refuse to do certain stuff - forcing us to go out in our Mustang. Their choice, not ours

"Safeguards and automatic fallbacks (beta): Fable 5.1’s biology and cybersecurity classifiers block fewer benign requests and now permit vulnerability finding in source code. Blocked requests return an error and are not charged to you. On the Messages API, opt in to fall back to another model so users get a response instead of an error. We recommend Opus 5 for biology and Opus 4.8 for cybersecurity. In Managed Agents, fallback is built in."

ImprobableTruth 11 hours ago | parent | prev | next [-]

Its "pure capabilities" are definitely worse than Fable, but I find codex has a much more pleasant style and is in comparison much more generous with its limits.

dmix 11 hours ago | parent | next [-]

Codex (+Sol) feels a lot more human for sure. Fable 5 is so, so wordy.

TuxSH 11 hours ago | parent | prev | next [-]

It used to be true up to 2w ago, but with the new/reinstated 5h limits I wouldn't be so sure anymore...

selectodude 10 hours ago | parent | next [-]

that's news to me, I'm still getting weekly limits, no hourly limits.

panos_news 11 hours ago | parent | prev [-]

Claude has a better 5hr limit?

versteegen 11 hours ago | parent | prev [-]

Ugh, people are still saying the Codex limits are more generous. They're not, Claude's are over 2x higher, have been for months! [1] It's just that Claude uses far more tokens, 2-3x is common. Except sometimes GPT will use just as many or even go into a compact loop and then your quota is gone, little headroom for hard tasks.

[1] https://devforth.io/agents-for-code/?sortby=monthly-value And I can confirm the numbers, I subscribe to both and watch the numbers

seaurchinzee 10 hours ago | parent | next [-]

That website seems to suggest that Opus 5 spends ~57 cents per task, while GPT 5.6 Sol spends ~49 cents per task? That ratio doesn't feel quite right to me. Artificial Analysis says Opus 5 High costs nearly ~3x as much as GPT 5.6 Sol High for a given task: https://artificialanalysis.ai/models/comparisons/claude-opus...

isoprophlex 11 hours ago | parent | prev | next [-]

No no our coffee is not more expensive! The serving sizes are just smaller!

trentor 11 hours ago | parent | prev | next [-]

I don't get it. It's the same result.

import 11 hours ago | parent | prev | next [-]

Well at the end of the day, I can finish more work with the codex limits.

re-thc 11 hours ago | parent | prev [-]

> people are still saying the Codex limits are more generous. They're not

They are if you follow Tibo on the resets.

re-thc 11 hours ago | parent | prev | next [-]

> IMO, Codex is worse than Claude with Fable.

Fable easily trips its safe guards. You can be 95% complete with the plan for it to trip and then lose it all. Anything is better than nothing.

enraged_camel 11 hours ago | parent [-]

>> Fable easily trips its safe guards.

Maybe it depends on the type of work you do, because for me it almost never happens.

>> You can be 95% complete with the plan for it to trip and then lose it all.

That's... not what happens though. The session will either seamlessly downgrade to another model mid-session, or it will stop with an alert and you can just re-prompt it. It will still have access to the context.

ipsod 11 hours ago | parent | next [-]

Web apps are where I have this trouble.

Making a web app secure is literally just finding and patching vulnerabilities, instead of finding and exploiting them. You could have the AI "try to make this app secure", find what it patches, and use it for exploits, and the AI can't know if that's what you're trying to do or not. I don't know how you can get around this. I get around it by not using Anthropic products, at present.

Exoristos 11 hours ago | parent [-]

Not to endorse OpenAI's particular guardrails, but unless you're doing something groundbreaking, security best practices should be more than enough for web development.

ipsod 10 hours ago | parent | next [-]

OpenAI is what I use most. Sol 5.6 still rejects a few requests a day when I'm working on web apps, but, overall, it's not too bad. I wish it'd auto-resume and try again, instead of waiting for me to intervene, but it's rare enough that it's not a huge deal.

It probably doesn't help that I'm using frameworkless PHP - I imagine a lot triggers could be avoided if I was using a framework where secure features were baked in.

re-thc 11 hours ago | parent | prev [-]

With OpenAI you can also apply for the security program, which doesn't require you to be a certified pentester (as per Anthropic).

kay_o 11 hours ago | parent | prev [-]

When doing basic CRUD apps I can count on fingers the amount of times guard rails haven't tripped and ended the session

boc 11 hours ago | parent | prev [-]

Small reminder that the US government rug-pulled Fable, not Dario. Lots of the safety guards that users find annoying/objectionable were the results of negotiations to get the model back online after the US government forced them to take it down.

Maybe Dario should have just "donated" $1M to Trump's inauguration fund like Altman, Meta, Amazon, Microsoft, Tim Cook, Elon, and Google. There's a reason they are the odd man out with this current Administration.

rudedogg 4 hours ago | parent | next [-]

Maybe Dario shouldn’t have tried for regulatory capture. He was constantly on the news talking about how these models are so dangerous and that we need regulation to keep China from releasing open-source models without guardrails.

sidrag22 11 hours ago | parent | prev [-]

The US government didn't make the choices to release the worst version of Opus and label it 5.0, and then isolate portions of their subscribers to limited usage of Fable.

They may have been unfairly targeted by the US government, but they are doing more damage to themselves without government help as well.

usef- 7 hours ago | parent [-]

Fable only being temporarily included in cheaper subscriptions was because anthropic is severely GPU constrained. They still are, and it impacts almost all of those unpopular decisions. They did announce from the beginning it was temporary.

sidrag22 6 hours ago | parent [-]

Horrifying excuse, gpu constraint can be used by all of these companies to justify a shit user experience. If the user isn't properly weighed in their priorities, they have their priorities setup wrong.

Their 20$ tier currently isn't serving their best model, and they insulted their users by putting out an ill tested opus 5.0, which is the worst experience ive personally had using a model in probably 2 years(obviously adjusting for expectations at the time of release).

usef- 6 hours ago | parent [-]

Yes, as a user you pick what works for you. But it is a reality for them that growth has been huge, and GPU manufacturing is bottlenecked.

People were very skeptical about how much investment most companies put into hardware/data centers two years ago, and anthropic was more conservative than OpenAI here, so it's potentially hurting them now.

(Opus is a separate story: it does seem to have improved in coding in my experience, most weirdness seems to be its human communication)