> We are probably going to need a lot more GPUs.

Or a breakthrough in algorithms etc.

The human brain, heck all bio brains, are proof that you don't need a lot of power or size for intelligence.

The human brain has 80 billion neurons and a 100 trillion synapses. I think you're underselling the processing power of that warm chunk of meat.

The real message of the last 15 years has actually been the opposite: if you throw enough processing power at it, intelligence emerges.

20 watts for inference AND training!

For intelligence, I expect the next breakthrough to be colocation of memory and compute in the same chip. And we'll need much more of this memory, probably a few petabytes.