Don’t know much about how distillation works so please enlighten me here.

> what are large language models but the distillation of all of the knowledge on the open Internet, scraped by the frontier labs and distilled into the models

If it’s as easy as that why do they choose to distill another model and not distill the knowledge on the open Internet from scratch?

known-good prompt-response pairs are more useful than random semi-coherent texts presumably

You need to do both.

A model trained on all knowledge from the internet (and other sources) is large but ultimately not very useful by itself, because it is going to spit out all kinds of garbage. You have to apply multiple further stages of training and refinement to the base model before putting it in front of users. So as an example you can train a model by yourself and then have GPT or Claude continuously check its outputs and correct it when it is wrong, ending up with a far more powerful model.

Because the model can output data in a manner optimized for training a new model, including outputs that were post-trained like RLHF and RLVR.