Totally agree, these kinds of problems are really common in smaller models, and you build an intuition for when they're likely to happen.

The same issues are still happening in frontier models. Especially in long contexts or in the edges of the models training data.