We're not going to be using tokens forever, and inevitably specialized hardware will solve this bottleneck. Underestimating how much proprietary advancements are loaded into Google TPUs is sort of like thinking the best we've got are Acura TSXs when somebody's driving around in a Ferrari.