> Are people this lazy?

It's not exactly laziness. It's that most people don't understand basic financial concepts like fixed versus operating costs.

Companies like Anthropic and OpenAI have invested huge sums in building/training models. Those are the headline numbers everyone talks about. What they don't understand is that the gross profit margins on AI model inference are very high.

The question is whether these companies ever get to the point where the profit from inference justifies (recoups) the investment made in training. If they have to keep investing vast sums in training new models, and/or competition causes the margins on inference to collapse, they're in trouble. But if training becomes less costly and inference margins don't compress too much, it's a totally different story.

▲

grey-area an hour ago | parent | next [-]

A few problems for Anthropic or OpenAI:

1. Model Build/Training costs are escalating if anything

2. They are in an arms race with competitors and can't afford to stop building new models - if they lose the perception of leading, they lose lots of customers overnight.

3. Inference margin is under attack from more specialised models and open weights models.*

4. They have no moat given how simple the interface to the products is - a bunch of text files and text prompts, and little prospect of building a moat.

So those all run counter the possibility of coasting on previous research. To me the most promising avenue for making money for these companies is to replace Google, not AGI or taking over the world's knowledge work. That would mean they are overvalued. Even coasting on previous research, how long would it take to pay off the money they have spent?

* https://strandsagents.com/blog/amazon-saves-millions-replaci...

▲

otherme123 an hour ago | parent | prev | next [-]

> What they don't understand is that the gross profit margins on AI model inference are very high.

Can we be sure of this? The second they tried to hike the prices to be profitable, they lost a lot of clients, or a lot of clients lowered they token expenditure. In any market margins tend to go to zero. I am using frontier models, but only because they are dirty cheap if not free. If they raise the price, I just switch to a cheaper model, start exploring the option of renting a GPU-VPS or buying my own card. Or reducing my AI usage, that currently includes stupid useless things like "translate this cooking recipe from french... for free".

> But if training becomes less costly and inference margins don't compress too much, it's a totally different story.

They are being priced as the "ifs" are certainties, and not only a possibility. If anything goes a millimetre below perfect for them, their finances collapse. High margins attract competitors, and unless you have a huge moat (and their only moat is expensive training), your high margins vanish.

▲

KaiserPro 36 minutes ago | parent | prev | next [-]

> gross profit margins on AI model inference are very high.

the gross margins on growing potatoes is very high, if you exclude all the other costs needed to get the potatoes to market.

Think about it, you take some potatoes you had left over from last yeah, dump them in the ground and wait 9 months. Boom pure profit.

This of course excludes the cost of the land, machinery needed to plant, till and harvest, storage, pesticide, fungicide and herbicide.

Then the transport, cleaning and packaging.

If you arbitrarily cut out all the bits of the business needed to make revenue, everything is a profit.

▲

surgical_fire an hour ago | parent | prev [-]

> It's not exactly laziness. It's that most people don't understand basic financial concepts like fixed versus operating costs.

> Companies like Anthropic and OpenAI have invested huge sums in building/training models. Those are the headline numbers everyone talks about. What they don't understand is that the gross profit margins on AI model inference are very high.

Only when you ignore that the models they provide inference for have an extremely high cost and pretend they don't depreciate. They keep having to train new models. I don't think they recovered the cost for a model when a new one releases.

It is kind of cute to say other people "don't understand basic financial concepts like fixed versus operating costs" when you seem to not understand basic financial concepts such as depreciation and amortization.

Also, and this is important, we don't really know their true compute cost. We don't know if their compute with the likes of Google and Amazon were initially subsidized, and for how long it might be. In Q2 they also got to not pay SpaceX for compute for a while.

	▲	ElProlactin 36 minutes ago \| parent [-]
		What's cute is that you're falling into the same trap of confusing accounting concepts. Gross margin is revenue minus cost of revenue (COGS). COGS is basically the compute that serves inference (rented capacity, or depreciation on owned GPUs used for inference, plus expenses like support, etc.). Model training is normally treated as R&D. Under US GAAP, R&D is expensed as it occurs and is below gross profit as an operating expense. It is not COGS. You mention "depreciation" but depreciation applies to physical assets like GPUs. The GPU depreciation that belongs in gross margin is already in COGS. A trained model would be an intangible asset, so that is amortization, not depreciation. Even that only applies if the training cost was capitalized, and under GAAP it generally wouldn't be. The full cost is expensed when it's incurred. Nobody is "pretending models don't depreciate." Their cost is counted all at once but not in gross margin because that's simply not what gross margin measures. Obviously, models lose value fast in an economic sense. They get replaced within a year or two and competition limits what they can earn. But that's an argument about whether the R&D expense is sustainable. It doesn't mean that gross margins are being misstated. "Inference has high gross margins" and "training is extremely expensive" can both be true, which oddly seems to be something many people have trouble comprehending. But this is the crux of the questions that will determine whether these businesses are viable long-term: 1. Will training costs fall over time, or at least grow more slowly than revenue? 2. Does the gross profit a model earns from inference before it's replaced exceed what it cost to train? 3. Will inference margins hold up as competition and open-weight models push prices down? You have a legitimate point about compute costs. If the deals with Google, Amazon or anyone else are priced below market, that directly inflates gross margin, because it lowers COGS. But we don't know those terms, so that's just speculation, and it's still a different argument from depreciation. To reiterate what I wrote: > The question is whether these companies ever get to the point where the profit from inference justifies (recoups) the investment made in training. If they have to keep investing vast sums in training new models, and/or competition causes the margins on inference to collapse, they're in trouble. But if training becomes less costly and inference margins don't compress too much, it's a totally different story.