hey, Building these from scratch in pure PyTorch is honestly the best way to deeply understand the paper details. something better than simply implementing a traditional Transformer or GPT-2，As an individual maintainer, will be able to keep up with future model updates?

I think depending on your perspective it might be even better to eschew PyTorch and implement with pure NumPy. Sure it will be slow, but perhaps as a pedagogical exercise it is more worthwhile.

will try my best, but open for contribution.