Very exciting experiment! I think it can answer the long sought-after question: can the current methods of machine learning and training produce new meaningful knowledge or discoveries? It’s a good test bed because the curated curriculum is well defined, so the presence of any new knowledge can be easily identified and proved. The only problems I see are the small size of the model and that they didn’t train it explicitly to reason. Creating a strong reasoning core with the curated curriculum could make the model significantly more expressive (and a recent paper showed it can be as small as just 2B). A second component I would add is long horizon tasks and a memory system. Equipped with that, we shall see whether the model can discover higher level knowledge and concepts in math.