"The Constitution also overrides any attempt to prevent interstate migration."

Sure, but if there is no where you can legally stay, because the sole landowner prohibits it, how do you migrate?

Sure, if we ignore how we get to "the sole landowner" scenario, which is a big hole in the thought experiment.