It’s all about the cache prices. Those dominate usage, cached tokens are easily >90% if not >95% of all used tokens.

Yes, but I'm not sure. They recently dropped cache prices 10 times. Why would they rudder back after only a few months?

_aavaa_ 13 hours ago | parent | next [-]

Don’t know but that’s not my point. My point is that even small absolute value changes in cached token prices will have massive impacts on final cost because of the sensitivity on it. If v4 flash cache prices go from 0.0028 to 0.01, the absolute change looks not that big (and would be a rounding error for input and output costs) but it would explode actual usage costs.

▲

faangguyindia 14 hours ago | parent | prev [-]

to fund training a far better and capable model.

	▲	HarHarVeryFunny 14 hours ago \| parent [-]
		Doesn't make sense - they very recently said they have plenty of money, and are only limited in training a larger model by lack of GPUs (Huawei as well as NVIDIA) available to buy.