But this assumes Chinese models will not achieve token cost optimization. Intelligence needs are fairly flat for many tasks, and the Chinese models have caught up on this front. Next they achieve greater token cost efficiency and we don’t need OpenAI.

▲

overfeed 12 hours ago | parent | next [-]

That the author doesn't acknowledge the relentless R&D efforts DeepSeek has been plowing into optimization, and giving a default win to OpenAI/Anthropic on the supposition that they've been serving models for longer is a black mark against the article.

I appreciate the transparency in explicitly stating their motivation for writing the article (a response to what the author saw as an overreaction to Chinese models), but I feel the article goes too far the other direction, with multiple unsupported leaps of logic, and overstating the stickiness of AI client products.

▲

VulgarExigency 14 hours ago | parent | prev [-]

The model that is most optimized around token cost is, in fact, Chinese. DeepSeek is astoundingly cheap by default, but if you use it from Reasonix (the harness optimized around its cache), it becomes even cheaper.

▲

ryeguy 10 hours ago | parent [-]

I keep seeing mention of the cache, what's special about it? All frontier llms have prefix caching, what is special about deepseek's approach?

▲

fspeech 7 hours ago | parent [-]

Their kv cache is smaller so they can use less vram and also keep your prefix cached for longer. https://deepseek.ai/blog/deepseek-v4-compressed-attention

▲

metalspot 5 hours ago | parent [-]

This also comes with significant capability reduction. deepseek-v4-flash is very good in the < 250K range, then degrades between 250-500K, and is practically unusable after 500K.

[edit]

This is my observation from using it without an specific context engineering to optimize for Deepseek's cache compression and sparse attention mechanisms. I am pretty sure that if you specifically structure your context to align to the cache compression boundaries you can significantly improve performance in the full 1M context, but there is not much reason to do this, because if you design your outer loop to work with shorter contexts that solution is portable and more efficient, so I haven't bothered with a optimizing for DS at this point.

▲

Der_Einzige 4 hours ago | parent [-]

Btw - assuming NeurIPS reviews aren’t garbage tomorrow, I’ll have a paper out which claims that most long context problems in models are really sampling problems in disguise

Switch to a modern sampler like min_p or ideally a better one like top-n-sigma (it’s in llamacpp) and your “my model gets stupid at long context problems” will basically go away.

Unfortunately this fact is still not well appreciated yet despite nearly every modern sampling technique getting an oral wherever they get presented. Min-K just got an oral at ACL 2026, for a hyper recent example of this. There’s a reason they keep getting orals.

The field massively ignored sampling for mostly safety reasons and now the whole field incorrectly believes long context doesn’t work on small models. Long context is an out-of-distribution problem. Your sampler configured properly keeps you in distribution.

Oh and this is doubly true for quantized models. I run my qwen 3.6 27b with 4bit quants from unsloth and get excellent performance because my sampler stack is good and not the garbage that is top_p and top_k. Also, yes, you need to ignore the trash recommended sampler settings from the Chinese labs (they’re wrong/bad).

	▲	metalspot an hour ago \| parent [-]
		Interesting. I am not familiar with model internals at this level because I have only been working at the application layer so far, but will definitely research this further. When you get the paper published would appreciate if you can drop a comment with the link so I can read it.