Cost of tokens goes down over time. Like by a lot. And it will continue to do so.

Imagine being in 2003 and saying compute costs won’t go down. That’s Ed lol.

EDIT: Some quick research on this so you guys have actual numbers: https://gist.github.com/dwaltrip/a037be938d2b5ecc8b8b238736e....

There's multiple separate angles that all contribute to token-costs going down: chip improvements, engineering improvements for running inference in general, AI architecture and training advances that give similar intelligence in a smaller model, improvements in the quality of the training data, data center design / economies of scale, networking and rack-level improvements that are multiplicative with chip advancements, and so on...

If you analyze the situation for 5 minutes, it's blindingly obvious that price-per-token will continue to improve. And there's a very similar case for intelligence-per-token as well.

And don't get me wrong -- I have many concerns about how this is all unfolding and how it will impact society. But let's get our basic facts straight.