We know that inference cost is very significant, as he shows for example in this piece.

https://www.wheresyoured.at/oai_docs/

However, it needs to be said that he received those numbers. I personally have quite a few issues with him, but there's no reason to doubt his journalistic integrity. Because of that, I believe he reports truthfully on data he receives by informants.

Additionally, none of the frontier models actually publicly talks about inference costs in anything but broad, "let's just forget that"-like takes. Which does not exactly spark confidence.

I'm eagerly awaiting anthropic's public disclosure of their financial details. That should be rather interesting in any case and finally put the inference-discussion to rest.

▲

remich 4 hours ago | parent [-]

No reason to doubt his journalistic integrity? He's not a journalist for starters. He's a PR flack who does PR for AI startups on the side while blogging on substack. There is every reason to doubt his journalistic integrity.

▲

lompad 3 hours ago | parent [-]

The PR-thing was always openly communicated by him and is not some secret or gotcha. It's essentially "fleecing the boosters", which I fully approve of and do similarly myself.

I'll gladly tell my customers all the most glorious stuff about AI and big tech while spending a significant chunk of the money they pay me on supporting AI-/tech-counterculture, such as doctorow, zitron and quite a few other writers, journalists and activists.

It's the old "you live in a society" counter-point against anti-capitalist activism. Needing to make ends meet does not imply that your points or principles are meaningless, it just implies that you have no interest in being homeless and that way losing your chance to actually change things.

So that's fine to me. But: I stated it for a reason, because I know others don't agree. I, personally, consider him trustworthy. You do not, and that's fine. I suspect we both await anthropic's Z.1, which will be able to settle a big chunk of the debate.

If he is right, the numbers will show it.

▲

simianwords 3 hours ago | parent [-]

Why do you consider him trust worthy when sooo many of his predictions are false?

https://news.ycombinator.com/item?id=48447549

	▲	lompad 3 hours ago \| parent [-]
		He was right about the cost changes, which he predicted quite some time ago. People shouted at him that he was making it all up - yet it was correct. He was also right about AI-video and sora in particular being a fundamentally flawed idea. He was also right about the dangers and problems with the general inaccuracy of LLMs and people relying on it. Also about the expected triggering of ROI-checking in companies, such as Uber is doing now. His prediction is, ROI is negative. And I'm awaiting the society's consensus on that. The general direction seems correct to me. He's not a technical guy and does not have the knowledge to critique models on a factual basis. I do wish he'd just focus on the stuff he _does_ know about, which is the financial side of things. He is a much needed counterweight to the unhealthy hype going around, imho.