The models are trained on data that encodes hundreds or thousands of years of turf battles and office politics. The model is shaped by the script of ‘Succession’, the history of the English monarchy and the Chinese Communist Party, and the biographies of Steve Jobs and Alexander Hamilton…

The models don’t generate what’s right and good: they generate what people are likely to generate.

They managed to fine-tune models to be good at math and coding, despite the vast majority of humanity being awful at those…

I dunno. You are probably right. But it’d be an experiment. It wouldn’t be an experiment if it was a sure thing.