| ▲ | The whole of PyTorch on one page(tensor.khalilli.ai) |
| 34 points by akhalilli 20 hours ago | 11 comments |
| |
|
| ▲ | bonoboTP 20 hours ago | parent | next [-] |
| Not bad, but Claude is not quite good yet for writing these explainers. We should try again in a few months with newer models. |
| |
|
| ▲ | Ask12j 20 hours ago | parent | prev | next [-] |
| Lol, a yaml file for function signatures of which to generate a giant switch statement. The "compiler" dynamo part is also a hack that calls real compilers like gcc and Triton. It is a big bowl of spaghetti. |
| |
| ▲ | akhalilli 19 hours ago | parent [-] | | the yaml thing is not a switch statement though. torchgen takes native_functions.yaml and generates the bindings, autograd nodes and dispatch entries from that one file.
there are ~~3700 ops. try keeping that in sync by hand. also dynamo never calls gcc. it just captures the graph from python bytecode. inductor is what writes triton/cpp and sends it to real compilers. they built their own compiler stack once (torchscript), it's dead now.
so this time they reused what exists. spaghetti yes. 10 years of layers. you can see every era in the repo. |
|
|
| ▲ | 4b11b4 20 hours ago | parent | prev | next [-] |
| Can tell it's garbage just from the first figures caption |
|
| ▲ | oliver236 19 hours ago | parent | prev | next [-] |
| can someone give recommendations of other pytorch explanations like this one? |
| |
| ▲ | akhalilli 19 hours ago | parent [-] | | author here. honest answer: there isn't much that covers the whole thing end to end, that's partly why I started writing this. but the ones I learned from:
- Edward Yang's "pytorch internals" post:
http://blog.ezyang.com/2019/05/pytorch-internals/
it's the classic, goes deep on the c++ side. - the pytorch developer podcast, same author. short episodes, one internal topic each - the pytorch repo's own CONTRIBUTING.md, the folder layout explained by the maintainers - and if you want to build autograd instead of reading about it, Karpathy's micrograd / zero to hero mine is planned as 12 parts, the list is at the end of the post.
part 1 (chapter Tensor: storage, strides, views) is in progress now. | | |
| ▲ | ayewo 19 hours ago | parent [-] | | Thanks for the PyTorch internals OG link and for putting together your PyTorch one-pager, if we can call it that :) Kudos. Off-topic: Some of your comments in this thread are dead, but I’ve vouched for some them as I don’t see anything wrong with what you’ve written, other than your writing here on HN sounds a little like you are using an LLM to compose each reply (or perhaps your regular use of LLMs is showing through in your comments?). | | |
| ▲ | akhalilli 18 hours ago | parent [-] | | thanks for the vouch, appreciated! :) and honest answer, English is not my first language, so yes, I rely on co-pilot tools when I write and it probably shows. the PyTorch content itself I can defend line by line, that part is mine. |
|
|
|
|
| ▲ | imperio59 19 hours ago | parent | prev | next [-] |
| If you like these sort of AI generated explainers this one is pretty good too: https://www.longformexplainers.com/llm-pretraining |
|
| ▲ | hnv9j3vudy 19 hours ago | parent | prev [-] |
| Solid write-up, thanks again |
