> The cost to serve tokens is absolutely profitable today

How can you possibly say that? Everyone knows that's not the case, these companies are losing money every day selling tokens. Revenue is not the same thing as profit.

▲

infecto a day ago | parent | next [-]

Don’t confuse what I say. Bottom line these companies are not profitable yet but it is profitable to serve a token via the API. They have increasing demand, not enough supply, models are getting better on quick timelines. For sure there may be some losers but it’s not hard to see that that token serving can be a profitable activity.

	▲	scrollop a day ago \| parent [-]
		This is a claim in the wind until you provide evidence.

▲

jeromegv a day ago | parent | prev | next [-]

Yep, especially if we look at what happened just last week, both Google and Anthropic have dropped how much you get out of your existing plans.

	▲	infecto a day ago \| parent \| next [-]
		Don’t confuse plan changes with profitability of inference. When people talk about the cost to serve a token and it being profitable we are referring to API cost not the plans which absolutely subsidize some level of use. Hard to know what is breakeven on plan math.
	▲	surajrmal a day ago \| parent \| prev [-]
		That's not a necessarily profitability thing as much as a demand thing. The only way to improve the supply for those willing to pay more is to take it away from those paying less. Once supply catches up to demand things will change

▲

dist-epoch a day ago | parent | prev | next [-]

There are private companies which rent/buy GPUs, run open-weight LLMs on them and sell the tokens. They absolutely make profit, and their clients think they get a good deal and are buying the tokens.

▲

naravara a day ago | parent | prev [-]

I think they’re losing money because they have to amortize the costs of training the models in the first place, which is where most of the resource sink is.

This is why they were freaking out about DeepSeek just taking the trained model weights and slapping an interface on it.

▲

malfist a day ago | parent [-]

Thats like saying a restaurant is profitable because they're making money selling meals if you ignore the costs of ingredients.

Of course they are profitable if you ignore their cost to bring a product to market.

▲

infecto a day ago | parent | next [-]

The problem with that comparison is restaurants largely don’t have much room to adjust price or optimize cost. The AI industry is too new with many unknowns right now so investors are willing to take risk. For the hyperscalers the bet is that being left out is going to be a greater loss than overbuilding.

▲

naravara a day ago | parent | prev [-]

That’s the wrong analogy. Model training is more like the setup costs of developing the menu and training staff. What’s driving the costs is important when talking about financial sustainability. If it’s mostly coming from optional R&D investments instead of the direct costs of producing the food then you can simply not exercise the option and be profitable. If it’s more coming as a variable cost that scales with each meal served that’s a very different situation.

Yeah it should be factored in, but it’s a different set of implications for long term sustainability. They don’t actually need to test and optimize a new menu every day or week. If they decide to just stick to the same one longer they can get way more return from each dollar spent on development. It’s just that right now the rate of improvement you get with training is really high and nobody can afford to fall behind their competition.

▲

malfist a day ago | parent [-]

These companies are continually training new models. This is not a long term amortized cost, it's actual COGS.

Yeah, sure you can ignore the cost of purchasing the building for the restaurant for most profitability calculations, but if every year or two you were tearing down your old building and building a new one, you better believe that has to be in your profitability calculations.

	▲	s1artibartfast a day ago \| parent [-]
		I think the relevant question to define the counterfactual is what would happen if they stopped training. If you can simply not remodel your restaurant and keep making money, then yeah, it makes sense to call it profitable.