If they don't control it, it's not the "other people's" land either.

Land belongs to whoever controls it. That's it. That is all it will ever be.

If there is not some higher power (e.g. the UN, who you say does not have authority), you have no recourse.

No matter what land it is or who they are: nobody currently living was there first. The only claim is always "I was the last to control it". But none of us are the first.