It’s way too early to tell the true cost of intelligence. Inference is still heavily subsidized, and training is apparently being funded by a mountain of free money

▲

infecto 2 days ago | parent | next [-]

How do we know inference is heavily subsidized? Are all these third party Chinese model providers subsidizing the true cost?

▲

cush a day ago | parent [-]

> How do we know inference is heavily subsidized?

Balance sheets

> Are all these third party Chinese model providers subsidizing the true cost?

It could be argued the subidies are even heavier than American model providers as the price war among Chinese providers is so fierce. https://www.scmp.com/tech/big-tech/article/3358868/after-tri...

▲

infecto a day ago | parent [-]

You did not answer the question at all.

Third party inference providers for Chinese models are in the US and there are so many players it would be shocking they are all heavily subsidizing it.

▲

cush a day ago | parent [-]

It would be shocking to you if in a market with many competing sellers would there be price competition such that those sellers are operating at a loss?

If you’re trying to get me to argue that every seller is operating at a loss, I don’t know. But overall the market absolutely is, again because balance sheets

▲

infecto a day ago | parent [-]

Balance sheets is not an answer. Sorry I am just asking for material proof. It would indeed be pretty shocking for every market participant, large or small, funded or unfunded to be operating inference at a loss.

Plans are definitely subsidized. Inference (token consumption) has never been proven to be operating at a loss and now that you can run many of the large SOTA models from China on sparks or other clustered servers you can figure out some of the math.

Not trying to argue but just saying “balance sheets” makes zero sense.

Yes lots of capex spend, hard to say if anyone has spent too much. At the same time demand is increasing for compute.

	▲	a day ago \| parent [-]
		[deleted]

▲

unknownfuture a day ago | parent | prev [-]

I don't think we have evidence that inference itself is subsidized, but I think we can assume the margins on inference are so low that the foundation model providers have absolutely no chance of recouping training costs and the whole thing is riding on a pile of VC cash and debt.

My bet is the assumption was one company would ultimately monopolize the space and then they could jack up rates. Alas, the opposite seems to be happening.