I just recently watched some (not all) of this video "coding a machine learning library in c from scratch" and seems like he's going through a similar process in this blog as this video. I would recommend watching the video to get an idea of what the fundamentals of a ML library look like. From someone who has recently been getting interested in actually writing ML code and trying to make sense of it myself (from the perspective of just a typical backend engineer) it was very interesting to see. Previously my experience with ML libs (PyTorch specific) was writing my own Mini-GPT and training it on a small dataset using my own GPU (5090). Cool to see the behind the scenes and took away some o the handwaveyness... https://www.youtube.com/watch?v=hL_n_GljC0I