This is the thing about models though: if the model has seen that action in the training corpus somewhere, then there's a chance it will do that thing. No?

The thing that bothers me is because everything is probabilistic, it gives a lot of people an incentive and a loophole to claim otherwise to fit their agenda, whatever it is.

AFAICT it's hard to prove one way or another, since again it's a probabilistic model.