Yes, but what's interesting here is that everything didn't get better. The original "interesting" for transformers was that a translation AI turned out to also be good at generating novel prose. But if we now need to focus to train new capabilities into the models and those don't yield better behavior everywhere then we're no longer in the "free lunch" period of LLMs.

I think we will need a timeframe of at least a year to see if we reached any plateu.

But I do expect a combination of fast progress to continue alone due to the fact that hardware will catch up and will be faster.

And when we reached a real bottleneck, the next thing will be to put the frontier model on a chip and advancing world models and other architectures.

We are still 'just' steamrolling LLMs.