I don't get the sky high valuation of LLM companies. I mean I get that these guys need a lot of money for compute to train the next generation of models. But Distillation does make it easy for other providers to replicate gains made by these providers at a much lower cost.

On a long enough timeframe, the open source models will catch up to the proprietary models and inference providers will beat these proprietary companies on price.

The high valuations are essentially lottery tickets, not something based on any sort of calculation of discounted future cashflows. The bet is that the researchers working for some of those frontier AI model companies will come up with innovations that give them a sustainable competitive advantage that goes beyond just purchasing more compute and licensing more proprietary training data. Obviously they can't all succeed but perhaps one or two will get lucky, perhaps by figuring out how to greatly improve efficiency or something that isn't easily copied.