Yes these things happen as part of RL Training. Same way that you can see the "But wait ..." phrases in thinking traces. They get rewarded.

Out of curiosity, how does something like "But wait..." get rewarded?