> There is no reason for the models to stop their improvements in the near future.

You speak as if "improvements to models" is just function of time, and resources are infinite.

Models keep improving as long as there are resources to allow for larger and larger datacenters, if we hit a scientific breakthrough once LLM technology become the bottleneck, if the economy is infinite to allow infinite growth, and (geo)politics is not a thing to worry about. Or we discover ASI, machine improve themselves and we reach the technological singularity.

I know everybody is drinking the kool aid by the gallon, but can we maintain a little bit of objectivity?

▲

AceJohnny2 2 days ago | parent [-]

yeah, it's funny how so many think the beginning of the S-curve is an exponential.

Granted, we don't know when the S-curve will inflect, but predicting too great an outcome is just as silly as discounting it altogether.

	▲	lumost a day ago \| parent [-]
		The s curve won’t inflect until it becomes difficult to allocate additional resources due to economic limitations. There is no sign that training a model on 10x the compute won’t lead to at least an equivalent improvement as the last order of magnitude increase. If we define the Pareto frontier’s input in terms of a magic “compute equivalent unit”. We get a free order of magnitude from nvidia hardware improvements every 2-3 years. We get another order of magnitude from capital expenditure every 6-12 months. Kernel improvements to the models themselves likely yield an order of magnitude gain at some periodicity.