Yeah if he had a random affair with a waitress or something, it'd be a bit scandalous, but not firing-worthy.

Sleeping with a Chinese spy or subordinate turns a personal problem into a workplace problem.