Competitive with opus 4.8 but weaker than sol or fable. About 20x cheaper.

xynelius 6 hours ago | parent | next [-]

If that wasn't impressive enough, it's actually ~60x cheaper if you take into account the typical cache-read/input/output split in agentic coding, and the deep discount for cache reads offered by DeepSeek. Opencode has some public data on the typical split [1]:

For DeepSeek V4 Pro the typical split is 750 in, 290 out, 82k cached.

Cost per request for V4 Pro: $0.000875 per request.

Equivalent Opus cost (w/o taking into account cache write costs): $0.052 per request.

[1] https://opencode.ai/docs/go/#usage-limits

▲

taosx 4 hours ago | parent | next [-]

I created a simulation for coding harnesses based on my own pi sessions. When taking into account all factors, DS-v4-Pro is cheaper than gpt-5.6-luna due to caching. Look at the bill segments difference for cache read cost and uncached cost between deepseek and the other models. At this point is cheaper to use ds-v4-pro than the luna models from openai.

ignore the numbers except the classic and keep in mind that classic is based on pi with the only change limiting tool output to 10kb

https://harness.eveid.com/lazy-harness-cost-simulation

* I built this for getting an initial estimate between different checkpoint/ compaction methods for the harness.

▲

RALaBarge 2 hours ago | parent [-]

Hey this looks good! Maybe consider adding a hover-over popup for the rectangles explaining what each thing means to a lay person. I see it at the bottom, but that is below the fold.

	▲	taosx 2 hours ago \| parent [-]
		Done, I'll take any other suggestions and apply them later, I will also split it a bit for different usecases as this was initially a throwaway prototype but found it useful. Basically it needs a bit more human touch.

▲

HDBaseT 23 minutes ago | parent | prev [-]

Can we have a conversation about subscription plans for a minute?

I don't mean to hype up the US AI firms, but if a ChatGPT $200/m subscription can get you $16,000 in effective API costs, doesn't effectively every model get destroyed by the subsidized Claude/ChatGPT models? Both in price and intelligence.

▲

JacobAsmuth 6 hours ago | parent | prev | next [-]

Per token. You need to look at pricing per task.

▲

trollbridge 6 hours ago | parent [-]

... which still comes out cheaper, since DeepSeek caches so much more.

I keep track of my token consumption even on subscription plans and my equiv. cost for my 5.6-Sol usage is around $4000-$8000 a month.

▲

dgellow 3 hours ago | parent [-]

How much do you pay for the subscription?

	▲	RALaBarge 2 hours ago \| parent [-]
		Not them, but I payed 10 dollars to DeepSeek directly to use their Reasonix tool. I worked all weekend and the past few days, billions of tokens, I still have 3 bucks left!

▲

swiftcoder 6 hours ago | parent | prev | next [-]

How does it stack against the updated Deepseek Flash version?

▲

pixelesque 6 hours ago | parent | next [-]

I've found Pro to be a lot better per "task" than the recently released Flash for code reviews and things (via OpenRouter running in pi.dev).

Flash makes a lot more initial mistakes, and then has to re-check stuff, and produces much more output compared to Pro. It often gets to the correct result eventually, but the output volume is often 5x more than for Pro, and the initial outputs are often wrong, with the first few saying something wrong (like there's a bug, or the code won't compile when it does), and then saying things like "Wait, let me re-check:", or "Actually, looking at it more carefully:" and then it thinks a bit more and eventually gets to the right answer.

▲

swiftcoder 6 hours ago | parent | next [-]

yeah, I've definitely noticed one has to be quite precise to keep Flash on the straight-and-narrow

	▲	RALaBarge 2 hours ago \| parent [-]
		Every plan and every code checkpoint finds me saying "Check with Grok and Fable latest to critique our strategy/code review" with pretty much every model. I havent ran into any deal breakers with the new Flash version yet (like it not running a tool properly or coming back with something completely daft)

▲

surgical_fire 6 hours ago | parent | prev [-]

I use a plan -> implement wotkflow for this reason.

pro plans, flash implements. I am super happy with how flash behaves like that.

▲

k__ 6 hours ago | parent | prev [-]

Around 5 percentage points better. (E.g., 87% instead of 82%)

▲

Gecko4072 6 hours ago | parent | next [-]

So not worth it over flash? Even at ~7x the size it isn't worth the price hike. Flash may be a monster of a model due to all the RL it received from free usage everywhere.

▲

networked 6 hours ago | parent | next [-]

I haven't tried DeepSeek V4 Pro 0813 yet. Recent experience tells me that larger models are worth it in non-obvious ways. MiMo-V2.5-Pro solved problems that DeepSeek V4 Flash 0731 couldn't solve for me: for example, adding a live counter for elided reasoning lines to a terminal-based coding harness. You wouldn't be able to tell from the scores on their respective Artifical Analysis page (https://artificialanalysis.ai/models/mimo-v2-5-pro, https://artificialanalysis.ai/models/deepseek-v4-flash). I like the DeepSeek V4 models, though. They critiqued my engineering decisions better than MiMo, and they seem to have a distinct aesthetic in the SVGs they write.

▲

trollbridge 5 hours ago | parent [-]

Interesting - I've been dropping into MiMo-V2.5-Pro-UltraSpeed whenever Flash seems to be "stuck" and it usually figures it out. I use UltraSpeed just because I'm so frustrated by then that I'm impatient.

I still find 5.6-Sol can solve some things neither of those can, but it's so slow (and it's so hard to trace / debug the reasoning) that I just let it run overnight.

▲

networked 5 hours ago | parent [-]

What about 5.6 Terra and especially Luna? Luna scores pretty high on benchmarks and seems to have different habits (like a denser pattern of tool use) and blind spots.

I'm trying out a development workflow where I generate mundane code with MiMo and Luna (and soon V4 Pro 0813?) and have Opus 5, which is running on only a Pro subscription, review and refactor it. I'm not sure it will justify the context switching, but it's an interesting exercise.

	▲	trollbridge 5 hours ago \| parent [-]
		Terra and Luna are fine, but they’re quite slow (OAI seems to be really slow lately) and don’t have the reasoning traces. My workflow really depends on them or I can’t switch models effectively.

▲

saaga 6 hours ago | parent | prev | next [-]

Yea that's what I was thinking. Flash is nuts. I find I have to be a more precise and specific with it but damn. It's crossed a threshold of production grade coding for sure.

I was running a session over a couple days and it didnt cross a dollar lol.

▲

npn 6 hours ago | parent | prev | next [-]

I still believe this is not the full potential of pro models. I expect they will release another checkpoint later this year.

▲

k__ 6 hours ago | parent | prev | next [-]

I tried the previous Pro model and in the end it was 50% more expensive than the previous Flash.

Wasn't worth it.

▲

eli 5 hours ago | parent | prev [-]

Opus 5 medium to Opus 5 max is only 3 points, if that puts it in context

▲

sparkling 6 hours ago | parent | prev [-]

deepseek-v4-flash feels so fast and snappy, i'm loving it. Happy to trade speed for the the 5% degraded benchmarking performance.

▲

saaga 6 hours ago | parent | next [-]

I feel the same too. I like the speed. I'm also a big fan of glm 5.2 fast. I can't wait for like 2000 t/s on these haha.

▲

k__ 6 hours ago | parent | prev [-]

I wouldn't exactly call it snappy, but faster than Pro, yes.

▲

ericd 6 hours ago | parent [-]

Single request depth on vllm with dspark, I'm getting ~200 tps, I'd say it's pretty snappy.

▲

JacobAsmuth 6 hours ago | parent | next [-]

Well sure but you're running on tens of thousands of dollars of hardware.

	▲	ericd 4 hours ago \| parent [-]
		It's much faster than other models on that same hardware in the same size class. I've tested a few, it's by far the fastest I've tested. And it wasn't tens* until recently. Didn't expect this to be one of my best performing assets this year.

▲

k__ 4 hours ago | parent | prev [-]

I get like 80.

▲

segmondy 3 hours ago | parent | prev [-]

... and mere mortals can run this at home or rent a GPU, you can't do so with Sol or Fable.