So much this.

The current-gen hardware is expected to have only 2-3 years of useful life. New tech, and especially new architecture is going to make inference much faster and more efficient in the near future, and the big vendors are going to be more limited by datacenter floor space and power/cooling, so that the hardware will be replaced very quickly by better models.

For the time being this thesis hasn’t worked out though. H100’s are only going up in price and becoming more scarce [1]. At what point would you update if now going on 3 years into H100 lifecycle this doesn’t seem close to happen.

People are buying v100s on ebay to run LLMs and Chinese manufacturers make custom PCIE boards for them. Older cards might not be useful for training but they're perfectly capable of doing inference work.

Jevons' Paradox. If it becomes more efficient, cost goes down, demand goes up, use rises, revenue rises.

Look at DeepSeek! They have done full system optimization on their model, architecting it so that it would be cheap to serve. I am proud to call DeepSeek a world class tech engineering firm in a way that 99% of the tech world is not. They are a 1% company.

> The current-gen hardware is expected to have only 2-3 years of useful life.

Expected by whom and why?

Google hasn't retired any of its now ten-year-old TPUs, or any of the subsequent generations. Maybe TPUs are special or Google doesn't understand compute economics but either argument requires data.

If your argument is depreciation, you're confusing a tax concept (depreciation) with useful life. Yes, they're supposedly related, but in practice, the relationship is rather loose. (Residential housing depreciates in 28 years yet typically lasts 2-3x as long and 4x is not rare.) Google's V1 TPUs

no, it's 10y. you just repurpose.

also, power and cooling agreements are usually 10y too.