When people talk about the commodification of inferencing, they imagine a future where everyone has access to frontier models and can run them at the same cost, and what will actually happen is closer to the commodification of _oil_, where only a few companies have the scale to produce it at a competitive price, and advances like this are _why_.

Once models are more or less interchangeable, the price of LLMs will drop to essentially the price of energy required to run them, and the big labs will be able to run them cheaper than anyone else.

▲

impossiblefork 5 hours ago | parent | next [-]

I don't agree. At the moment companies like NVIDIA take several times what it costs to make a chip. I think the fair split for the technology contribution is more like 50-50, maybe even 30-70 in favour of the manufacturer.

With competition we will actually have the fair split, whatever that is, and thus much lower prices.

At the moment, to have a big AI firm, or really AI firm at all, you need to be blessed by NVIDIA, in the form of receiving circular financing for your compute. They know that their prices aren't fair, or competitive.

Commoditization of inference is the end of that. The end of the mega-premium on inference hardware, and it's good not only for people who like running their LLMs, but it's the first step towards commoditization of training.

	▲	kubb 5 hours ago \| parent [-]
		10-90 is the fair split.

▲

simianwords 7 hours ago | parent | prev [-]

I don't believe models will be commodified because each model is unique with strengths and weaknesses. Its not like Steel which is more or less the same no matter where you purchase it from.

If what you said were true, you would hardly see people complaining about the quality of Opus 5 or good writing from Sol. But people do.

	▲	zurfer 6 hours ago \| parent \| next [-]
		The same level of intelligence gets roughly 10x cheaper per year. So you might both be correct where a large part are commodity tasks but frontier is hard and valuable and not commodities.
	▲	lelanthran 6 hours ago \| parent \| prev \| next [-]
		> I don't believe models will be commodified because each model is unique with strengths and weaknesses. They are all converging.
	▲	airspresso 7 hours ago \| parent \| prev \| next [-]
		This depends heavily on what the use-case is. Yes, if it's a coder making software and having to read LLM output then writing style matters. If the LLM is used in an automated data processing pipeline with a capped level of complexity, entirely different aspects matter and LLMs become more interchangeable.
	▲	skhameneh 4 hours ago \| parent \| prev [-]
		> Its not like Steel which is more or less the same no matter where you purchase it from. I’m not an expert in metallurgy by any means, but this seems really off. There are many recipes for steel and varied processes that also impact the final product.