The biggest discovery might actually be that they ignored regular caching rules and kept invalid cache suffixes and it didn't hurt performance

I wonder how bad the performance would be if they plain ignored the whole rotary encoding dance and just back-filled precisely the parts of the cache that changed directly. Would it break the model? Confuse the model? Or would the model internally correct for it?