This is a really good question.

What convinces me is this: I live in SF and have friends at various top labs, and even ignoring architecture improvements the common theme is this: any time researchers have spent time to improve understanding on some specific part of a domain (whether via SFT or RL or whatever), its always worked. Not superhuman, but measurable, repeatable improvements. In the words of sutskever, "these models.. they just wanna learn".

Inb4 all natural trends are sigmoidal or whatever, but so far, the trend is roughly linear, and we havent seen seen a trace of a plateau.

Theres the common argument that "Ghipiti 3 vs 4 was a much bigger step change" but its not if you consider the progression from much before, i.e. BERT and such, then it looks fairly linear /w a side of noise (fries).