Is this not the heart of the IRM paper by Arjovsky, Bottou et al.?

Oh theirs is a far more sophisticated and deeper idea, to look for invariant representations.

Mine is a quick but effective practical hack fuelled by a little bit of insight.