It's quite clear that these companies do make money on each marginal token. They've said this directly and analysts agree [1]. It's less clear that the margins are high enough to pay off the up-front cost of training each model.

[1] https://epochai.substack.com/p/can-ai-companies-become-profi...

▲

m101 4 hours ago | parent | next [-]

It’s not clear at all because model training upfront costs and how you depreciate them are big unknowns, even for deprecated models. See my last comment for a bit more detail.

	▲	simonw an hour ago \| parent \| next [-]
		They are obviously losing money on training. I think they are selling inference for less than what it costs to serve these tokens. That really matters. If they are making a margin on inference they could conceivably break even no matter how expensive training is, provided they sign up enough paying customers. If they lose money on every paying customer then building great products that customers want to pay for them will just make their financial situation worse.
	▲	ACCount37 2 hours ago \| parent \| prev [-]
		By now, model lifetime inference compute is >10x model training compute, for mainstream models. Further amortized by things like base model reuse.

▲

emp17344 an hour ago | parent | prev | next [-]

Sue, but if they stop training new models, the current models will be useless in a few years as our knowledge base evolves. They need to continually train new models to have a useful product.

▲

magicalist 5 hours ago | parent | prev | next [-]

> They've said this directly and analysts agree [1]

chasing down a few sources in that article leads to articles like this at the root of claims[1], which is entirely based on information "according to a person with knowledge of the company’s financials", which doesn't exactly fill me with confidence.

[1] https://www.theinformation.com/articles/openai-getting-effic...

	▲	simonw an hour ago \| parent [-]
		"according to a person with knowledge of the company’s financials" is how professional journalists tell you that someone who they judge to be credible has leaked information to them. I wrote a guide to deciphering that kind of language a couple of years ago: https://simonwillison.net/2023/Nov/22/deciphering-clues/

▲

9cb14c1ec0 5 hours ago | parent | prev [-]

It's also true that their inference costs are being heavily subsidized. For example, if you calculate Oracles debt into OpenAIs revenue, they would be incredibly far underwater on inference.