Why does Britain have any say in the people who live in, say, Liverpool? Because that's historically been British territory. So what's the difference? Only real difference I can see is length of history (a few hundred years vs. nearly a thousand years). But as far as I'm concerned, a few hundred years vs. nearly a thousand years doesn't make much difference. I'd argue that any territory that has belonged to country X for longer than all of its inhabitants have been alive has a pretty fair claim to be historical territory of country X, and should continue being part of country X unless there's a very good reason otherwise. (Such as a valid treaty, a clear referendum, and so on).

It gets all complicated and messy when war is involved, of course. I'm talking about peaceful transfers of ownership here.

So you’re suggesting amnesia and ignoring the fact that Gibraltar wasn’t a blank spot on the map before it was annexed by the British. I think that’s a very one-sided view.