This is likely true. I think model quality has stagnated and that its likely a non-trivial task to find a new improvement vector. Scaling the width of the model (which has been the driving force behind the speed of improvement thus far) seems to have reached its limit.

It will be interesting to see the implications of this. Tooling can only do so much in the long term.

▲

mxwsn 7 hours ago | parent | next [-]

How do you know that width scaling has been the driving force of improvement?

	▲	waterTanuki 2 hours ago \| parent [-]
		I mean, it's not exactly a PhD level question. One can infer from the extreme demand of GPUs and DRAM + new data center construction that all the providers are banking on width.

▲

4 hours ago | parent | prev [-]

[deleted]