There's a long standing line of thinking that data compression has something to do with intelligence, or that intelligence may "just" be data compression in some sense.

In the case of models, the almost-theory (hypothesis with a good deal of evidence) is that once the amount of training data exceeds what the model can simply represent internally, the best way to keep improving is to start to "understand" the data. Understand in this case means to form internal control structures and decision systems that represent the data algorithmically rather than just memorizing it.

I put understand in quotes because we don't know how far this goes toward what we call understanding, or whether there's still a lot missing.