Remix.run Logo
▲ rcxdude 20 hours ago

Just to be clear, the difference between the older models and the newer models in terms of mathematics ability is not from having trained on even more papers, but by reinforcement learning on the act of proving theorems. Anthropic now spends about as much on this training phase as on the 'pretraining' with the masses of scraped or scanned data. This doesn't mean it's not still essential, but it's far from the entirety of the result either.