There is a Sanjeev Arora paper "A Theory for Emergence of Complex Skills in Language Models" (https://arxiv.org/pdf/2307.15936) on this subject. The key idea is there is cross entropy (how "surprised" the model is with the "correct" next token, lower is better), some of which is inherent in the language and therefore unavoidable, and the rest is model error, and that this portion of the cross entropy is reduced with scaling.

And as scaling reduces a model's excess entropy, the model can become good at combinations of skills much faster than you would expect if it had to separately see and memorize every combination. They call this "slingshot generalization".