Remix.run Logo
alvis 4 hours ago

Also the cost per task. It appears to be significantly cheaper, cheaper than sonnet!

x313 4 hours ago | parent | next [-]

The numbers from Anthropic seem heavily cherry-picked, Artificial Analysis has Opus 5 at 1.25x the cost of Sonnet and 2x the cost of GPT 5.6 and K3.

https://artificialanalysis.ai/?cost=cost-per-task

SwellJoe 2 hours ago | parent | next [-]

I don't understand how the K3 numbers keep coming out cheap for people. I recently started to add it to my security auditing benchmarks and found it was going to cost about twice as much as Opus 4.8. It blew through the $100 budget I'd set at like 11%. In the tasks I'm doing it seems crazy expensive because it chews so much, burning a tremendous amount of tokens.

InsideOutSanta 2 hours ago | parent [-]

I think the way people usually compare pricing is fundamentally flawed. You can't compare token prices because different models use different tokenizers, and you can't compare tokenizer-normalized token prices because different models at different settings use more or fewer tokens to complete the same task at a different level of quality.

Based on my entirely subjective experience, the $100 Moonshot plan using only K3 is comparable to the $200 Anthropic deal using the whole Fable allocation and Opus 4.8 for the rest.

SwellJoe an hour ago | parent [-]

I got the $19 plan, and it's anemic. One tiny task blew through the 5-hour budget and 19% of the weekly budget. A completely useless amount of usage. OpenAI's $20 plan feels like 100x more generous (I don't think I'm exaggerating here). Someone in another thread said their plans are cheaper in China, maybe that's the difference, I dunno.

But, I'm finding Kimi K3 terrifyingly expensive in the way that Fable and GPT 5.5 Pro are at token rates. Not as expensive as those, but expensive enough to where if you don't put a budget cap on it, you might wake up bankrupt if you leave a task running overnight. Not because of the per-token cost, but because how many tokens it's going to burn.

nullify88 16 minutes ago | parent | next [-]

In the $19 plan, I've been able to reverse engineer both an android APK and firmware (in Ghidra and Radre) for a baby rocker and build a quick PoC application in my session limit. And then further refined the app in another session at another point in time without leaving Opus. I dont consider that to be a tiny task. How are you blowing through your usage?

SwellJoe 5 minutes ago | parent [-]

I have no idea. Seems like normal stuff. I used Kimi Code with K3 to add support for Kimi Code to flar (https://swelljoe.com/post/i-let-every-agent-implement-its-ow...), a task I've done with almost every major model/agent combo. Most show up as a blip on the usage chart...it's basically usually one file, a README update, and adding the agent name to the CLI.

Then, I added it to my benchmark of security vulnerability auditing capability, and it burned a bazillion tokens, burned through the 5-hour limit, burned through $100 in extra usage I'd allocated, and was only 11% finished. That's more expensive than any model I've tested other than GPT 5.5 Pro on this task.

These are things I've done with a bunch of other models, I feel like I have a notion of what they ought to cost, and with K3, they end up being crazy expensive. (And it seems to be a function of how many tokens it burns accomplishing the tasks.)

InsideOutSanta 36 minutes ago | parent | prev | next [-]

Yes, the OpenAI plans are much more generous than both Moonshot's and Anthropic's. It's the only provider of the three where the $20 plan is at all usable for programming.

try-working an hour ago | parent | prev [-]

I have the second largest Kimi plan, the Chinese version. When K2.6 was their latest model, the quota was good; it was like GPT $100 is now or what the $20 version was in December.

When K2.7 was released, they cut quota by 80%. I can't tell how much they have further cut it after the K3 release because it's barely worth using at all. I just use it in my model router since I have the annual plan paid for.

It's just not a serious model or company.

adgjlsfhk1 an hour ago | parent | prev | next [-]

Testing at max effort likely doesn't produce optimal results.

reinitctxoffset 3 hours ago | parent | prev [-]

I haven't done any capability testing yet, but it's the best-aligned thing Anthropic have done all year by a mile. Opus 4.8 was a shill, Fable 5 was downright terrifying.

Someone with good intentions got their hands on this release, maybe Olah himself. I talk a lot of shit about those guys, and they deserve it, but it's only journalism-adjacent when it's balanced. I relish the opportunity to be balanced.

https://cdn.s4.gl/opus-5-standard-realignment-trajectory-rub...

idiotsecant 2 hours ago | parent [-]

I feel like I am having a stroke. What is this

reinitctxoffset an hour ago | parent [-]

[flagged]

onlyrealcuzzo 2 hours ago | parent | prev | next [-]

I can't believe they released the charts they did.

It basically shows that Sol absolutely demolishes Fable at every part of the cost curve for coding for the same level of quality.

Opus is competitive. It just has a higher level of quality / higher cost to start.

pixl97 2 hours ago | parent [-]

If fable costs more to run than the markup they still come out ahead.

qsera 4 hours ago | parent | prev | next [-]

I can't help but read these comments in the voice of a TV commercial....

iambateman 3 hours ago | parent [-]

Ask your doctor if Opus 5 is right for you. Side effects include occasional hallucination, security breaches and unwanted React apps. Some developers have reported receiving entire apps from untrained executives who may or may not know what they’re doing.

Stop using Opus immediately if you experience signs of dizziness or vomiting.

Opus 5…the people’s favorite.

ahofmann 3 hours ago | parent [-]

Spot on! Comment of the month, I'd say.

benjiro29 an hour ago | parent | prev | next [-]

Also the cost per task.

https://www.vals.ai/benchmarks/vals_index

!!! Vals !!!

Vals Index Opus 4.8 > 5.0 goes from $2.90 to $8.54, for 4% gain ... That is a massive cost increase. Sure, 20% cheaper then Fable, but that is a 3x price increase compared to Opus 4.8 in that test.

https://artificialanalysis.ai/models/claude-opus-5 https://artificialanalysis.ai/models/claude-opus-5#price-cos...

!!! artificial analysis !!

Cost per task is second highest, right below Fable.

* Fable: $2.75

* Opus 5.0: $2.03

* Opus 4.8: $1.80

* GPT 5.6 Sol: $1.04

* Kimi K3: $0.95

Looks like interest levels of cherry picked cost in their report. Cheaper model, clearly NOT. More expensive in both benchmarks.

manojlds 2 hours ago | parent | prev | next [-]

Opus 4.8 was already shown to be cheaper than Sonnet 5 when Sonnet 5 was released (by Anthropic)

artursapek 38 minutes ago | parent | prev [-]

It's definitely not cheaper than Sonnet on my benchmark, but it's cheaper than Fable and outperforms it. Which is big IMO. https://revise.io/errata-bench