> At 20 billion parameters per chip, you’d need just 50 accelerators to support a trillion-parameter model

I don't see any evidence that this is possible. From my understanding, the whole model needs to be on a single chip. Which rules out any popular frontier models with several trillions of parameters. Even smaller sub-frontier models have hundreds of millions of parameters, so these would be ruled out as well.

The methods for splitting weights across multiple chips are well established. Groq/Cerebras can't hold a model on one chip either.

I think it's enough that a single layer fits on each chip if you can daisy-chain them with good interconnects.