> Why would knowing about 2019 internet memes help you in any way at coding?

99.99% of the knowledge an LLM has is useless for a given scenario, the hard part is knowing what the .01% that’s needed is. Knowing as much as it can means the model can handle edge cases, turns of phrase, etc.

Put another way, it avoids overfitting. That’s basically the insight that’s given way to the current AI boom.