GPUs are the land that's staked. They're of limited intrinsic value in the sense that they're in the way of getting to the gold (models), but there's a small supply of it so they go for a lot on the market. But if there was literally any other way to make the models even for a few percent cheaper, they model builders would move to that. On the inference side, most of the cloud providers are looking for pretty much any way to server that up more cheaply, with custom TPUs, or other tensor units of some type.

We saw this with crypto mining where truckloads of expensive GPUs were dumped in the trash after the proof of work became so hard it became not worth the cost of electricity to keep on that generation of card.