That's the argument authoritarians use when they want to censor something. The problem with it is that it proves too much. Do we also get to apply it to symbols of communism because of the millions of people who died under Mao, or the US flag because of slavery? What about that book Marx wrote that led to all the horrors under the USSR; can't displaying books also be symbolic?

You don't want the government to have the power to decide things like that. It's better that they censor nothing.