Maybe... I'm not enough of an expert on the financials to say, but it seems to me that inference should be able to recoup the cost of SOTA infrastructure, unless you then also use a large portion of that infrastructure to train new models. And I also think the race to remain SOTA itself is also largely a function of training, because my understanding is that training benefits more from the leading edge of hardware.

But yeah, I definitely don't have high confidence in any of this!

Yes, that makes sense. I don't have "the" answers either :-)