There is something called "wrapping/unwrapping" - you encrypt the key, move it to the other HSM, and it unwraps it. So what moves around is encrypted. I don't know enough about HSMs to tell you if it's done using symetric or asymetric crypto, but no the key should not move around in the plaintext.

The write-up from eMudhra mentions something about wrapping/unwrapping ceremony failing.