> If token costs converge toward zero for most AI use cases...

In the real world, token costs seem to be going up, as early stage pricing at a loss gives way to pricing that generates revenue.

Compute costs might go down a little over the next five years, but there's nothing coming along in hardware that leads to huge reductions in price. NVidia says don't expect better price/performance before 2030.

The models keep getting bigger, and people put loops around them which iterate, burning tokens.

Where is this cost reduction coming from?

I've switched to non-SOTA models, which deliver comparable value at a fraction of the cost. A full day of coding with Deepseek is approx $1 in tokens, and at least for my use cases the quality is equivalent to Claude.

I don't expect any of the third party openrouter providers sell tokens at a loss. Agreed that increasing model size could drive token prices up however so far there's been a very strong trend in the opposite direction with smaller models becoming increasingly capable thanks to advances in theory and implementation.

Edit: A glaring omission on my part there is that growth of aggregate industry demand for tokens has the potential to outpace increases in supply provided by new datacenters buildouts. So tokens certainly could go up depending on how things play out.

Don't index on tokens. Costs for useful unit of work had gone down in my experience, and that's what actually matters.

Counting tokens is like counting lines of code.

The following two things can be true at the same time:

- Frontier state-of-the-art performance keeps getting more expensive (and better).

- Any fixed performance level is becoming cheaper.

(And a third: if you still want to see improvements instead of a fixed level, you can trail the frontier a bit and still see price some reductions over time.)

This website https://tokenpriceindex.com/ tracks Token Cost.

You are right - tokens are going up currently.