This is just temporary though, right?

With the benefit of LLMs already being proven, in a couple of years we will have vastly better hardware for inference I guess.

I feel like now hardware is stagnating a bit, because the software side has moved too fast for the hardware to catch up. Once we settle on sone good, optimal software architecture for the models, dedicated hardware will easily increase throughout by 10x or 100x, for a fraction of the most.

LLMs seems quite simple, maybe we'll be able to print at home our own chips with the models.