In the US, anti-immigrant sentiment is often the highest in regions with relatively few immigrants, far away from the border. The less that immigrants are part of peoples daily lives, the more people seem to fear for their "culture".

Presuming to have any real agency over the evolution of human culture also seems misguided. Culture is like a river, in that "no man ever steps into the same one twice". It's constantly changing in unpredictable ways, from generation to generation or even within generations, with or without immigration. That goes for the cultures of immigrants also.