How could that possibly work? Deepseek was undercutting every other provider by an order of magnitude on cached tokens.

Do they just set a super low caching time and hope that drops effective cache rates low enough? Do all other providers somehow overcharge by that much? Are they just going to sell it as a loss leader?

▲ lcampbell an hour ago | parent [-]

> Do all other providers somehow overcharge by that much?

This, I think. Cached inputs have an opportunity cost (keeping the KV cache until use) but a hit is basically free. “Basically” - if the cache is offloaded to system RAM or NVMe there’s some scheduling overhead.

From a consumer viewpoint a more interesting metric than the raw costs is

    cached cost * hitrate + input cost * (1 - hitrate)

from a personal standpoint rather than a per-provider one (e.g. if OpenRouter is blindly dispatching your requests you might have a bad time).

	▲	skeledrew 23 minutes ago \| parent \| next [-]
		> if OpenRouter is blindly dispatching your requests This can somewhat be the case, depending on your config. I updated mine to make DeepSeek high priority because I was having a lot of cache misses and reliability issues with the default (cheapest (at face value)) providers, and cost was actually higher overall than anticipated. Was smooth sailing from then; might have to tweak things again now pricing has changed though.
	▲	Tharre 42 minutes ago \| parent \| prev [-]
		System RAM and/or NVMe storage still has a real cost. And swapping out the context between VRAM and system RAM / NVMe still consumes bandwidth. I don't have a clue on what the real cost to inference providers comes out to, but it seems really weird that there would be such a big gap, in what should be a pretty competitive market.