Question out of curiosity. I once swapped a TPLink media converter between two homes, both using the same ISP, to debug internet issues and to see if that would improve the situation. Did I do something incredibly illegal? And did my ISP get confused seeing my media converter on the other side of town?

This wouldn't be criminally illegal anywhere unless done with some sort of fraudulent intent, but maybe in some places the ISP could make you swap them back.

Illegal? No, at least not in any sane jurisdiction. It's no different than moving a SIM card between phones.

Confused? Maybe but probably not. It depends on how they track things. An ISP I had in the past tagged subscriber accounts on the OLT side.