What's interesting to note is that the "intelligence" labs can squeeze out of an H100, an almost 4 year old GPU, is dramatically higher than what they got out of it in 2022.

It hints that once these labs get a good enough "everyday model", they can work on efficiency so they can serve these models on old hardware. Which is almost certainly already happening.