That "TPU advantage" might be slowing Google down (though likely not as much as their internal bureaucracy).

Porting CUDA-based research, debugging, and overall experimentation speed is likely slower.

The GPU is still king for training.