Tokens too cheap (to meter), by Simply RAG (1982):

  I been laid off from work.
  My cloud hosting is due.
  My dev team all needs
  Brand new roles to pursue.

  So I went to the boss
  To see what he could do.
  He said, "Son, looks like ChatGPT
  Got a hold on you."

  Tokens too cheap (to meter).
  I can't compete with a neural-net reader.
  Oooh, tokens too cheap (to meter).

> Tokens become cheaper than tool calls

The author observes that a call to GPT-5.6 Luna is only 4-5 orders of magnitude more expensive than grep, and then predicts that at current rates of progress, calling an LLM will soon be cheaper than a grep. I think this is a good time to invoke Stein's Law: "If something cannot go on forever, it will stop." These efficiency improvements won't continue forever. It's more likely that the per-call cost of high-quality, compiled software like grep will be a lower-bound that LLMs asymptotically approach, rather than a line that they blow past with perpetual exponential progress. (Barring a true breakthrough in something like quantum computing or room-temperature superconductors.)

I found the OP insightful and worth a read. Thank you for sharing it on HN.

The only aspect that is poorly analyzed by the OP is business model viability. All players are investing insane amounts of money in infrastructure with the expectation that their future profits will justify all that investment. The winner or winners in the AGI race, they believe, will find the proverbial "pot of gold at the end of the rainbow."

The OP glosses over questions of business model viability with a brief qualitative discussion and very little hard data. For example, to earn an annual return > 10% on every trillion dollars of capital sunk into infrastructure, the owners of that infrastructure must earn free cash flow (operating profit less investment) in excess of $100 billion per year in perpetuity. Is that feasible? Why? How?

The OP does not really consider such questions.

A much deeper analysis on the falling price per task was published yesterday by Epoch AI [1]. It's a real statistical analysis and comes to more defensible and grounded conclusions. The headline takeaway is:

The cost of a given level of performance often falls fastest right after that level is first achieved, that is, when it is state of the art (SOTA). We see this pattern on three of our five main benchmarks of AI capability. Averaging across all five, cost falls 66% per quarter (75× per year) for performance that has just debuted as SOTA. Two years later, prices fall half as fast, at 32% per quarter (4.7× per year).

but the analysis itself has more nuance and is a quite interesting read.

[1] https://epoch.ai/publications/the-plunging-price-of-thought

> NVIDIA will still boom

I think Nvidia is under the same pressure as Anthropic/OpenAI. Nvidia will dominate research and probably keep dominating training, but the real volume is in inference. And for inference Nvidia's lead is only a few months, similar to the lead frontier labs have over open source. Nvidia will sell a lot of Rubin CPX's, but their margin on that will be a lot smaller than B200 because there is so much more competition in that space.

GPU case doesn't seem that strong? The graph is impossible for me to reason about at least. You could draw basically any trend line through that GPU graph and it would look equally plausible to me. The main takeaway I get is that the NVidia H100 from four whole years ago is barely different in efficiency from the state of the art, which is surprising to me, and seems to indicate the exact opposite of what the article says.

There seems to be a mistake in the cost comparison between 2025 and 2026. The 2025 chart axis is the cost to run the entire "intelligence index", and the 2026 version is a weighted average cost per task.

I don't disagree with the thesis here, I just don't think costs are coming down quite that quickly.

for the first chart (sourced from https://epoch.ai/data/machine-learning-hardware?view=graph&y...), what is the audience supposed to think about that trend line? there's a step after you slap a regression on some points where you evaluate whether there's a real trend or noise, right? i don't see that either in the article or the linked source.

It is hard to see how the environmental side effects of this aren't going to be somewhere between bad and disastrous.

>We are likely to see LLMs running locally at current frontier-quality on commodity hardware in the next 3-6 years

Yeah okay bud, anyone checked in with the state of consumer hardware recently? Not the author, evidently.

>oh in 3-6 years this will all be over

Yeah I'm sure Samsung, Nvidia and sk hynix will all be very calm with lower volumes and lower margins.

This is the core of my belief that data center construction is a huge bubble.

AI is not a bubble, IMO, though we may see a retrench and some companies with sky-high valuations will crash to more reasonable ones. But data center demand is probably a bubble, and the main driver will be reduction in the actual amount of power and data center space required to serve escalating demand.

I think hardware and model improvements will pace or maybe outrun demand and then when demand starts to saturate will keep going and leave a lot of orphaned data centers.

So this week's coordinated shock and awe campaign is about "open models" and "prices are falling rapidly". Also on the front page:

https://news.ycombinator.com/item?id=49815526