It was because the new people were not white. They were black. The culture was not a factor, no matter how respectability politcs exactly as white they acted, they were black and thus a threat.

That's what 'FuriouslyAdrift' is saying. Those are the people he/she does not want in the neighborhood.