Moore's law stalled ~2015. Unfortunately, no way current models will run on the <100W thermal budget of a cell phone. Printing the weights directly into a chip would help efficiency a lot, but not enough.

In all likelihood in a few years we'll get ~200-400bA~4-6 MoE models that are on chip, and they'll be better than the current frontier.

Is it conceivable that in 10 years time we’ll have 7B models that have the same level performance as modern frontier ones?