But there simply aren’t enough black people in positions of power to enact racism against white people on the kind of grand scale it currently operates at against black people.
This is what white society fears the most, that if they give up their power they will be treated the way black people have been for centuries. They recognize the harm caused and don't want it returned.

