> Probably because they colonized white people.

Better to say that they colonized non-African or non-American indigenous people who could not be conveniently attached to vehement Trans-Atlantic chattel slavery or Catholic-conquistador narratives, because the Khanty, Buryats, Yakut, etc. have an appearance that is hardly what most people using race-based terminology would associate with “white people”.

They also colonized Finns, Balts, Ukrainians, Belarusians, Poles, etc.