Why couldn't an LLM, if it was smart enough, generate and consume its own data?

I know the answer: because it leads to model collapse. But why is that? Wouldn't a smart model not collapse? It's seeming like they keep getting smarter because we keep pouring more of our own knowledge into them, not because they are actually getting smarter. And yes, sometimes a dumb but persistent bruteforcer can make new discoveries.

> if it was smart enough

and i think this is exactly the crux;

the really big models need really big datasets

and current gen LLMs get a lot of training data beyond "all books + all of the internet"

the objection is then that producing this additional data would already confound it with pre "virtual cutoff date" knowledge (since the training data probably implies mathematical and SWE concepts that were developed post "virtual cutoff date")

If it is smart enough to generate data it can consume to train itself better, it is already smart enough to not need to do that.