You hit the nail on the head. It's a race to the bottom.

Everyone is raising the bottom. Kimi got 60% more expensive during the 2.x cycle despite staying the exact same size.

Now K3 is almost 6x the cost of the original K2 checkpoint, and while the parameter count finally jumped, it's still an extremely sparse MoE and definitely does not cost 6x what the original K2 checkpoint did to host at scale.

Race to the bottom only takes real effect when there's a cap to the capabilities, otherwise everyone races to the bottom of a rising target (how economically valuable the tokens are)

▲

NikxDa 10 hours ago | parent [-]

But Kimi (at least say) will release their weights. So surely, if their prices are too high, somebody else will host it cheaper?

	▲	BoorishBears 9 hours ago \| parent [-]
		a) Why and b) With what compute? "Why" as in, why take lower margins when Moonshot currently can't service all the demand for the model anyways. Based on past models no one is going to massively undercut Moonshot: few have the chops to serve it as efficiently as Moonshot and of those few, most of them don't go for being the cheapest, they go for being fast + reliable (think Together, Fireworks). You get what you pay for applies very much with how many axes there are to serving these increasingly large models. - And for "with what compute": as the value of a token goes up, what people are willing to pay for compute is going up. Every once in a while I'll see a story about falling rental rates, but with even slightly more established clouds I've been seeing availability get worse and worse over time. I'm pretty sure the only reason the highly informal indexes don't reflect this is because every neocloud trying to cash in on an NVIDIA Inception discount kicks off by selling unrealistically cheap compute for a bit.