I certainly get what makes you feel that the model should have been wise enough to utilize the knowledge in its pretraining, but it seems like the pretraining data gets pretty muddled in later stages.