Not a core paper, but I found Formal Algorithms for Transformers [1] (a Google paper from 2022) to have a great pedagogical style.

[1] https://arxiv.org/abs/2207.09238

I'll add in here https://web.stanford.edu/~jurafsky/slp3/, "Speech and Language Processing", with chapters that deal specifically with LLMs and transformers.