I do not (generally) trust benchmarks. I only trust what a model does with MY code.

Forget DS. I asked MiMo 2.6 yesterday to explain ML/LLMs to me succinctly and the pointed it at Karpathy's micrograd code. It produced a C implementation called `xor_mlp`, a tiny model that learnt how `xor` worked. I then asked it to produce a model that can play tictactoe without losing (mostly). It did. It supervised the training process and produced a compiled version with multiple switches. The pi-dev session is still running, so here are actual stats

↑45k ↓35k R1.0M CH99.4% $0.019 4.2%/1.0M (auto) - (opencode-go) mimo-v2.6-flash • high

And here is Luna on the same workflow (I had to poke and prod a bit to get what I wanted):

↑141 ↓34k R1.0M W43k CH95.3% $0.072 4.2%/1.1M (auto) (opencode-go) gpt-5.6-luna • high

I expect similar results from DS41F/MS13. Closer to MiMo costs than Luna.

So the "significantly cheaper" thing may not really hold, more so when Luna has to actually read my codebase to do the stuff that I want rather than rely on world knowledge. The 8-10x cache read cost differential itself will kill the token budget.

Artificialanalysis benchmark is a combination of a several benchmarks which might or might not represent realistic coding:

"Artificial Analysis Intelligence Index combines performance across 10 evaluations: AA-Briefcase v1.1, GDPval-AA v2.1, AutomationBench-AA, Terminal-Bench 4.0, SciCode, Humanity's Last Exam, GDP.pdf, CritPt, AA-Omniscience, and AA-LCR v1.1."

Not saying it doesnt have any value but it's probably irrelevant if you use these AIs for a specific use case. Like for example Humanity Last Exam tests general knowledge, which is not very useful for coding.

It's best to go to the specific coding benchmarks and compare there.

According my model,

My cost is (I use nous as provider)

DeepSeek v4-flash-0731 • Your cost: $0.56

DeepSeek v4.1-flash • Your cost: $1.22

GPT-6 Luna • Your cost: $4.22

My usage is heavy on the cache. Apparently v4.1 flash uses 1.75 times as many tokens so still cheaper.

Does it use less tokens or we just get no accounting of the thinking tokens in OpenAI/Claude models?

I am not designing rockets. Most of my work is bog standard hobbyist stuff: compilers, vms, sandboxes, system tools of various kinds, SSGs, markup languages, plain text ledgers etc. Even Gemma/Qwen running locally can manage this.

Frankly, I have no idea what people do with Opus/Fable etc. I don't think anything I do needs something that charges $50/M for output tokens.

I have used MiMo 2.5 extensively. MuseSpark and DS4 Flash are MUCH smarter than that one. But MiMo follows instructions diligently. So it has been useful as the implementer of a spec designed by Claude/Kimi.

One good thing about MiMo that I experience on OpenCode is the provider seems to cache tokens for much longer than MS13/DS4F. I have seen cache being hit for close to an hour after the last request. The corresponding timing for MS13/DS4F is in the 1-5 min range.

I am trying out MiMo 2.6 Flash as well.

Last I heard, caches had like a 5 minute TTL... doesn't that mean if you get up and make a coffee (hand pour over of course), that you are back at full price?

I have written about my experience. I have also mentioned the kind of code I write. It is not react/js/css heavy stuff that I see a lot of people write. So the code bases are typically in the 5-50KLOC range. Freestanding C, Python, or maybe some TypeScript. And fairly modular. I can thus run models on specific modules without having them read everything into context.

So the workflows I mention work for this kind of stuff.

Are they bleeding? Their multipliers seem to be reasonable. They are not offering $60 worth of usage for $10 on every model, only some. In the case of the expensive ones, it is only $15.

Given how subscription models work (not every one uses every last $ of their plan), they should achieve breakeven soon enough I guess.

They already stopped. That's why the service quality declined.

I gave you the $40 option. Which is what it would cost if you used APIs on OpenRouter or elsewhere. Still beats Luna by 4-4.5x

MS13 is pretty sharp and has been my workhorse for the past month. It follows my coding style and commit/clean workflows referenced in AGENTS.md perfectly but has the habit of doing things without conferring with me (the Gemini problem). So you need some kind of instruction for that.

It starts failing around the 5-600K context mark, but you can have it generate a handover document and continue in the next session.

I would not use it at sticker price, but the Contributor version is priced just about right.

shocking. the code it generated, while technically working, was entirely garbage. i used it for code review and it flagged twenty issues, sol checked the review and found 75% of them were hallucinations. sol was much closer to reality. i no longer trust benchmarks at all because of it.

You can use the models I mentioned directly from DeepSeek, Meta and Xiaomi and not exceed $40. Were it not for GLM 5.3 blowing up a quarter of my monthly budget in 5h, we are actually looking at something like $30.

Chinese buy their tokens at home. West as a market is an afterthought for their companies them. Western AI is banned, so only used via resellers by small fish, not companies. US has zero presence at that huge market, and absolutely not a moat.

They are buying Huawei accelerators in bulk to serve their local customers. The whole system is currently optimized to deliver a lot of cheap LLMs and hardware for them to run on.

> Luna is about suppressing inexpensive Chinese model competition.

I think so too. To me the so-called Chinese local models are a clear move to prevent US companies to establish a foothold and build a moat around their business. US companies are clearly invested in a strategy to make themselves relevant with claims of major impressive achievements with the so called frontier models, and how these and only these are unblocking whole ranges of applications. At the same time, they are heavily invested in pushing AI on all absurd types of mundane tasks, such as transcribing meetings and... talking to your own kids?

In the meantime it's rather obvious that, in spite of all the propaganda, frontier models are required only in ultra niche applications, whereas the ability to run any model at all already provides most of the value. In fact, US companies have been renownee by dumbing down older generation models in what seems to be a desperate attempt to make newer models look better and influence their uptake rate.

So there is no better way to take the wind out of the US AI companies' sail than pulling a two-punch attack consisting of not inly releasing capable models that refute the "only US frontier will do the job" thesis but also releasing them for free to commodities them and eliminate the business impact of dumbing down models.

*govt bailouts

Flash thinks much more so it’s pretty much line with Sol for performance. That said I like flash coding style much more than OpenAi models.

TBH: I really like how fast 3.8 Flash is... Once I have clear plan, I feel quite confident in delegating large parts of implementation to Flash and Luna

look at token use, 3.8 flash is a huge token hog compared to openai models

Gemini 3.8 Flash and 3.1 Pro are pure rubbish. Very little thinking, mediocre and usually incorrect results. They cannot be compared to frontier models.

Despite the error in the parenthesis, it's exactly what he says: https://artificialanalysis.ai/?intelligence-category=text-on...

There's no difficulty in cancelling.

Wait, so you guys don't anonymize the user data, then train on it after it's been sanitized? I thought this was done to some degree or another.

So what is the value prop then? Just basic supply and demand?

FWIW I have definitely noticed OpenAI's emphasis on efficiency and value in the last year, so that part isn't new to me... I just thought there was more to it then that.