H100 is nearing five years and costs more to buy a used one now than a new one when it was released :)

You are completely missing the bet these companies are making.

They think can outlast their competitors and capture a larger portion of the pie while the cost of inference keeps going down dramatically.

If you haven't been paying attention, the cost is about 1/100th of what it was in 2024. This is the trajectory pretty much every technology has followed.

Of course there will be market crashes and corrections and things like that and most companies won't survive, but the bet is that whoever survives ends up doing pretty well.

The fact that supply constrained GPUs holding value is negative indicator. It's like the tulips mania except bubble enthusiasts are buying wilted/dead tulips because live tulips supply constrained due to irrational demand. Now GPUs has more gross utility than tulips but seems like at current revenue/capex spend, every GPU is still negative net financial yield - they lose money - literally buying tulips and watching it wilt. Economically, better off simply not buying and losing more. That's the level of economic irrationality at place sustaining bubble, at least for hyperscaler/big tech balance sheet - small operators logic different and antagonistic to big operator demand/business model.

If cost of inference goes down 100x, would need 100x more demand. This makes overspending on GPU even more irrational. Jevons this, Jevons that but ultimately irrelevant. At end of day, leading players, hungergame winner candidates is saddling themselves with so much debt, even if they survive, post crash they are immediately uncompetitive against new entrant with blank slate and newer gen, more efficient GPUs that will be cheaper to buy/operate post crash when hardware prices will revert to mean.

It doesn't matter if some of the current players survive, they've basically stabbed and weakened themselves so much any healthy upstart in the future can wipe them out unless they lock in legislative protection... safety regulations, ban open source models etc.

That is the new bet, regulatory capture moat, because economic bet is entirely lost, especially with open models eroding mote.

You just stated yourself that it costs more now used than when they were new.

If everyone's running local then why are these larger companies dumping cash into data centres?

>H100 is nearing five years and costs more to buy a used one now than a new one when it was released :)

Because everyone is buying as they want to run their own models and not pay for a cloud service?

This is absolutely the calculus. There is no moat. It is survival of the best financed. Open AI and Anthropic are in very precarious situations.

Isn’t that the same bet that famously profitable companies like Uber did in the ride share market?

What costs are 1/100th?

> They think can outlast their competitors and capture a larger portion of the pie while the cost of inference keeps going down dramatically.

We are also within an arms race of training newer larger models with more speed while discontinuing older models.

Gemini/Chatgpt have already discontinued their models from 2024 (iirc) because they are using all their compute in serving/training newer models. Being quite frank, nobody is serving a model from 2024 as the intended use-case while having very little moat as open source models are catching up.

> Of course there will be market crashes and corrections and things like that and most companies won't survive, but the bet is that whoever survives ends up doing pretty well.

How so, by raising the prices? because the current prices aren't sustainable and I feel as if there would certainly be companies which will try for one reason or other to be cheaper to capture the market share because of the larger promise of whoever is able to get as market share. I had once thought about it and I don't think that even in an ideal world, they would end up doing pretty well given no moat.

Also even if a company survives and ends up being one of the survivors and makes profit in the ideal scenario you mention, then within some years other companies will try again and construct more datacenters and end up driving the prices down for everyone, so nobody knows how things might look down for 2-3 years let alone a decade, so I remain a bit skeptic currently so.

I had actually thought some on the economics of datacenters and I found it to be very related to power. The only ones which seems to be making money might be the power generators actually because power is the actual bottleneck rather than GPU's in datacenters from my understanding.

Though the power is raised at the cost of electricity bill increases for everybody including people living in houses. The job prospects are minimal as well, as a nation, aside from just getting investment just for the sake of it because AI's trendy right now, I feel like its a net negative deal for people living there.