I think theres still lots of room for huge jumps in many metrics. It feels like not too long ago that DeepSeek demonstrated that there was value in essentially recycling (Stealing, depending on your view) existing models into new ones to achieve 80% of what the industry had to offer for a fraction of the operating cost.

Researchers are still experimenting, I haven't given up hope yet that there will be multiple large discoveries that fundamentally change how we develop these LLMs.

I think I agree with the idea that current common strategies are beginning to scrape the bottom of the barrel though. We're starting to slow down a tad.