> if you have the right to travel between internal states

The relevant cases are Corfield and Paul [1].

They restrict states. Not the federal government. And since 1926, “the presidential administration” has explicitly had the power to “deny or revoke passports for foreign policy or national security reasons at any time.”

[1] https://en.m.wikipedia.org/wiki/Freedom_of_movement_under_Un...