Architecture difference wrt vanilla transformers and between modern transformers are a tiny part of what makes a model nowadays