> Would you called majority Japanese now a colonizers since the originally come from Korea/China and before them they were people there?

Depending on context, yes, especially considering that (AFAIU) there still exist identifiable (socially, not just genetically) ethnic groups on the Japanese archipelago who predate that colonization event, and who still experience forms of ostracization typical of such colonization. There'd be no cognitive dissonance for me because I refuse to internalize a definition of colonialism that tacitly presumes European exceptionalism and supremacy through a sort of reverse White Man's Burden logic of moral accountability and historical criticism.

For the same reason, I recognize that groups we (i.e. westernized, globalist, cosmopolitan, what-have-you types) typically call aboriginal in a homogenizing, undifferentiating manner were often colonizers themselves thousands of years ago, displacing other aboriginal groups that may or may not still exist today. There are multiple such groups in Southeast Asia. And the first such modern human aboriginal group may have colonized an area occupied by pre-modern, archaic humans. (Or possibly vice versa!)

Buying into the logic of modern anti-colonialism critical theory is not required to appreciate and criticize the harms European colonization inflicted and continues to inflict. But rejecting that logic might be a prerequisite to recognizing and appreciating the exact same dynamics and harms that played out and still play out today among non-European ethnic groups.