Upvoters cannot see removed comments. You can try it here [1] yourself to see the effect.

It's also possible for moderators to add a username to a subreddit's automod config to silently remove all their comments, and that would be a "shadow ban" from that subreddit.

[1] https://www.reddit.com/r/CantSayAnything/about/sticky