The point is that them thinking something is racist doesn't make it racist.
But even if they are wrong, their argument is still that racism is unethical. You can't correct a misinterpretation by another misinterpretation.
The point is that them thinking something is racist doesn't make it racist.