I misread the title and thought it would be about the (for lack of a better term) NanoGPT speedrun[1]. Which, previous to the article, was meant to be the world speed records for Andrej Karpathy's GPT-2 (small) reproduction.

1. https://github.com/KellerJordan/modded-nanogpt#world-record-...

You're not the only one. I thought so too.

I just ran it the last couple days extensively to verify my data training pipeline I'm building for my gonano SIMD port.

Given that the speed records and the runs are sponsored by the same company I was confused a bit.