Yes, there are people who aren't white who do bad things- nobody denies this. Chattel slavery was largely a white Anglo-American invention, though, and the aspect of treating the children of enslaved humans as the property of the people who claimed to have the right to own their parents is a uniquely American invention.

And no, we don't have to have a world in which people of all other races must be saints for critiques of the white power structure and its role in the last five centuries of human history to be valid.

IOTW, saying "other people did it, too" is no defense.