It’d be interesting to think if device swarm serving of these models ever becomes possible - are a few projects that seem to be just starting on this.

It seems more plausible to me than wherever the frontier is at any given time getting small enough to serve on one local card directly