Conclusion: both are true which makes sense. The KV cache scaling yields both the emergent power and requires the enormous capacity.

▲

JKCalhoun 13 hours ago | parent [-]

Which does sort of hint at a (power/profitability) ceiling on the LLM line of AI… That should make the industry nervous.

▲

ACCount37 12 hours ago | parent [-]

Does that follow at all?

High end AI is at its most useful when you use it to replace high end human labor. You can't buy 9000 cybersec specialists on demand, but you can buy more Mythos tokens.

Then we get into all the scaling curves. Such as: LLMs getting more capable per FLOP, per byte of weights, per byte of VRAM, etc. And: inference compute getting cheaper over time.

I see a lot of "should make the industry nervous", but when you try to dig into it? It's wishful thinking, every fucking time.

	▲	waynecochran 11 hours ago \| parent [-]
		As the article states, right now Anthropic does not have the compute capacity. I suppose they could charge an enormous amount of money, but if it is indeed that powerful there are folks that would pay and they would degrade everything. To make matters worse, bad actors could use it to find zero-day exploits in the power grid and banking system.