Diversity doesn't make places more dangerous (if i understand the stats). But humans are naturally tribal and fear those who look and act significantly different.

That depends on an arbitrary perception of 'my tribe'. Irish and Italians used to riot against each other. Germans were hated (look up a quote from Benjamin Franklin). Protestants and Catholics used to riot. Every wave of immigrants seems to get the same treatment by some.

Within a few generations, their decendants marry each other.

Because humans are tribal they will also go on to attack and prey on those who are outside their tribe, making diversity more dangerous. Especially when diversity is not merely some people of different ethnic/racial backgrounds living and working together, but a population split into isolated cultures with different circumstances.

Unless there's a big strict enforcer to keep everyone in line of course.