Right but if you go into a country then you're in the country, not in the outskirts. You still pay taxes (generally...), and, in many countries, don't get any social services.

If anything, many formally-colonial countries are leeching off their illegal immigrants, not the other way around.