| ▲ | bayesianbot 7 hours ago |
| That is exciting! I don't understand how DeepSeek can be so cheap with their cache pricing - ~0.003 usd / 1Mtok. 100x less than Kimi K3, or similar numbers against pretty much any other decently sized model to my knowledge. I've been using it whenever possible as even longer agent sessions cost few cents. |
|
| ▲ | sudosysgen 7 hours ago | parent | next [-] |
| If you read DeepSeek's papers, you'll find a litany of architectural features that allow for a greatly reduced cache hit price by shrinking the size of the KV-cache. |
| |
| ▲ | yfontana 6 hours ago | parent [-] | | How come no other big model seems to be able to deliver the same type of extremely low cache cost though, if their techniques are public? | | |
| ▲ | jboss10 6 hours ago | parent | next [-] | | I think the "architectural features" are part of the model, not the kv cache. So implementing it would be difficult and expensive. | |
| ▲ | petu 6 hours ago | parent | prev | next [-] | | Deepseek V4 paper is just ~three months old | |
| ▲ | sudosysgen 6 hours ago | parent | prev [-] | | Many of these techniques haven't been published very long ago - it often takes a good 6-8 months for techniques to percolate. But also, they come at a complexity cost and, seemingly, also at a stability cost. | | |
| ▲ | hnfong 3 hours ago | parent [-] | | Also potentially a performance (in terms of output quality) cost. DeepSeek is cheap on a per token basis but lags behind in the benchmarks, perhaps it was a calculated tradeoff. |
|
|
|
|
| ▲ | hack1312 7 hours ago | parent | prev [-] |
| What provider are you using? |
| |
