I don't think this is strictly true. Train a model that can't output 2 + 2 = 4, but can output 3 + 3 = 6, and so on... That'll cost you. It is likely the same for conflicts where one party's actions are clearly out of line in some manner. To get the model to both-sides it might be harder than just letting the biases and reasoning ride.

A model that understands it might be wrong with the token it predicted after “a + b = “ and is open to double checking that with a calculator/script is a more intelligent model that is less likely to hallucinate confidently.