What happened to your grandparents was wrong. I'm sorry it happened.
It happened- and I'm assuming virtually all of it was inflicted by people with white skin-because they were immigrants, and because they were followers of a minority Christian communion-not because they were white.
And yes, if whites are actually going to claim they have it just as bad as people of color on receiving hatred and bigotry, it DOES need to be comparable to what black people have faced. This matters because white people, in talking about things like this, ALWAYS imply that what they've experienced is somehow worse than anything black or brown people have experienced, and entitles them to be indifferent or dismissive about what black and brown people have gone through and CONTINUE to go through- and to oppose measures that would allow black and brown people to move towards economic equality with whites- such as measures to make it easier for black and brown people to purchase homes, which is something they need to be able to do to achieve a middle-class standard of living.
If white people were to use past suffering as a means to nurture their own empathy, and to let go of the toxic, pointless "any gain for somebody else HAS to be a loss for us" mindset, that would be fine. But people of our skin color, instead of that, use whatever past hardship we might have suffered simply to justify giving ourselves an exemption from empathy and suffering and awarding ourselves the right to judge others.