Colloquially racist? Sure. Anyone can be, but racism is also a political structure which black people don't have access to in the US. Which is what OP is trying to get at.