For a 32 bit number you're looking at going from using 256 to ~1800 transistors in the operation itself. A modern core will have roughly 1,000,000,000 transistors. Some of those are for vector operations that aren't involved in a xor or sub, but most of them are for allowing the core to extract more parallelism from the instruction stream. It's really just a dust mote compared to the power reduction you could get by, e.g., targeting a 10 MHz lower clock rate.