Today the closed-source giants maintain a substantial cost-per-token advantage due to proprietary inference tricks like speculative decoding kernels, ultra-optimized FlashAttention derivatives, multi-GPU pipeline orchestration and dynamic prefix-caching algorithms.

I'm not sure whether the DeepSeek's open-sourced inference engine does all of these, but it appears that plenty of inference tricks have just been revealed for the open source community to learn from and start improving upon.

▲

torben-friis 3 hours ago | parent | next [-]

So... this makes it easier for someone in the position to run chatgpt class models as a service to get similar returns to openAI/anthropic, if I got you.

And I imagine this engine is by default mostly just deepseek compatible and inference engines are not universally applicable to all models?

It sounds significant for competition but I'm not sure how it sinks Anthropic/openAI. Presumably they still have their frontier models, the training data they're vacuuming from current users and access to hardware to serve at a ridiculous scale, right?

	▲	schneehertz 3 hours ago \| parent \| next [-]
		DeepSeek can accomplish over 90% of tasks of equivalent value at one-tenth of the fully loaded cost of O/A, and this capability can further spread through open source. In my view, the IPO risk for O/A is no longer just a valuation discount, but a substantive challenge to its long-term business model as a high-margin foundation model provider. In that scenario, even if global Token usage increases 100-fold in the future, that alone would not be enough to justify these foundation model companies being worth one or two trillion dollars.
	▲	mobelkh an hour ago \| parent \| prev [-]
		I think for me it's more the possibility of the ideas here being incorporated in other Opensource projects (vllm, llama.cpp, or in the architecture choices of the models themselves) Deepseek manages to be ridiculously cheap and efficient with prompt caching, i spent 550M~ (1.9b input cache miss, 1.7b output) tokens and it cost me 5~ bucks, that would be ~150 bucks on Opus 5.5, if i ran that workload in parallel that would be $50 vs $1500, e.i. i wouldn't want to subagent heavily with anthropic models. now imagine Kimi K4 or GLM 5.5 with those cost savings

▲

asdf88990 3 hours ago | parent | prev | next [-]

I think you’re correct, although suggesting that open AI is a trickle drown of AI companies is a bit naive.

But the key point is that something good enough always attracts more interest and contributions because a productive outcome is very realistic and tangible.

Open Weight models and competing open source inferences engines massively undermine monopolies; in theory at least.

See how cloud is easy to divest and “nationalise”.

▲

FeepingCreature 2 hours ago | parent | prev | next [-]

All of those are in open-source engines. I'm not convinced they have that much of an advantage.

▲

CurbStomper4 3 hours ago | parent | prev [-]

[dead]