I have moved between clients (as a contractor) within the same industry every couple of years for nearly two decades, often cycling back around to the same clients more than once.

There have also been cases where a previous client kept my contract open after I had left and started with the next client. This gets disclosed to the new client, and my previous client gets informed of where I’m moving to so they have the option of immediately closing the contract if they feel there is a conflict that cannot be managed.

None of this is hard. It simply requires transparency and not doing dumb or nefarious things.