Tbh, if the model is small enough you can train locally.

I don't need my code assistant to be an expert on Greek myths. The future is probably highly specialized mini llms. I might train a model to code my way.

I'm not that smart enough to figure this out, but the solution can't be to just brute force training with more gpus.

There is another answer.