I expect by that point we'll have local models that can do a decent job, I would guess give it a decade and we'll be running custom accelerators that are smarter than current frontier models.

In the same way that only supercomputers used to have multiple processors and caches but it's now standard.