Why are hosting providers charging to much to host it, compared to much larger models? https://openrouter.ai/compare/qwen/qwen3.8-27b/deepseek/deep...

2 things, 1st: Alibaba's official endpoint pricing. they don't want to undercut too much as there is profit to be made to be close to it but not too low

2nd, and maybe more importantly: KV is not as efficient (vram usage-wise) as something like deepseek v4 flash. for 256k, fp8 kv is 9.3gb (full precision ~17.3gb). deepseek v4 flash is ~2.5b for the same size at full precision (which is fp4/8, if you are interested in it, read the paper, its pretty cool).

Doing the math, hosting 27B at NVFP4 (~23gb) with 2.3M total ctx (9 agents) matches the vram usage of ds v4 flash for the same 2.3M ctx (2.3 agents). the break point is 1.5M (6 27B agents) if you use full precision 27B.

To be clear, the qwen3.5 architecture (what 3.8 uses) is still considered decent in terms of KV efficiency, its just that dsv4f's architecture is SOTA in that space, and with the lower active params, you get better max kv scaling and higher speed serving that, if you have a lot of gpus.