It was because the new people were not white. They were black. The culture was not a factor, no matter how respectability politcs exactly as white they acted, they were black and thus a threat.