It may not matter. Think about why SOTA model companies are exploring chips. What do chips offer?

If SOTA models haven’t peaked, then the SOTA model companies would still be churning out better and better intelligence.

Google rolled out TPUs in 2015. AWS released Inferentia and Trainium chips in 2020.

If companies working on ML-specific chips was evidence that large transformer models have fully saturated their potential, the field would have been done circa GPT-2.