Author here. You’ve just summarized the main part of the article. To keep things simple, the focus is on pure facts. But yes, the outcome of next token prediction is much more profound than wrong facts.