the moat always has been and will be compute

and we are fast approaching limits which will be hard to overcome - electricity, chips

Don't you think that we will eventually get more specialized hardware that will greatly improve efficiency? Running neural networks on GPU:s seem like quite wasteful?