I really can’t stand when writers point to the difference in price per token on the api and subscription and use that as evidence that inference loses money. This author even says it’s implausible that the api charges 4x marginal cost when I think it’s very likely even higher than that. The entire rest of the post sits on this faulty assumption. Fixed costs don’t matter when marginal revenue is profitable and growing rapidly. The ai labs only have 2 questions. Can they prevent users from switching to open source models? Can they scale the number of users on enterprise plans the way they did for coding but in a more general way for all knowledge jobs?

▲

jimbokun 5 hours ago | parent | next [-]

Then what are the real costs?

▲

martinald 5 hours ago | parent [-]

Wrote this a while back. https://martinalderson.com/posts/no-it-doesnt-cost-anthropic...

OpenRouter is the best guide to real costs.

	▲	anthonypasq 3 hours ago \| parent \| next [-]
		agreed, this doesnt even account for prompt caching or the fact that anthropic has substantial proprietary efficiencies on their inference stack specific to their models and scale.
	▲	jimbokun 4 hours ago \| parent \| prev [-]
		Thanks, that’s exactly what I was looking for! And much more informative than the speculation and guessing in the article.

▲

bcjdjsndon 5 hours ago | parent | prev [-]

> Can they scale the number of users on enterprise plans the way they did for coding but in a more general way for all knowledge jobs?

Do these knowledge jobs have a significant corpus of not only knowledge but discussion and problem solving, all conveniently labelled for the AI to train on? Probably not. Coding has stack overflow, what does, say, advertising use?

	▲	HDThoreaun 5 hours ago \| parent \| next [-]
		I agree this is a hard problem for the labs. I would be hesitant about “probably not” though. There is just as much marketing copy floating around as there is coding training data. I struggle a bit in this question because I’ve only ever worked as a software engineer, so I can’t exactly make claims about all the work other jobs do. But, one example is I was talking to a doctor friend of mine the other day. He was talking about how he had to take his recertification exam recently and put the questions into chatGPT and thought it gave answers that were generally more thoughtful and correct than his own. Does that mean doctors are done? Of course not, but he’s now pushing hard for more ai tool use in his practice.
	▲	warkdarrior 5 hours ago \| parent \| prev [-]
		> Coding has stack overflow, what does, say, advertising use? Advertising has centuries of print ads, 100 years of radio advertising, 70 years of TV commercials, etc. And modern AI does not necessarily need labeling.