The problem is the people who tap you on the shoulder, or worse... those who know they shouldn't disturb you so hang quietly around your desk until you give up and make eye contact or otherwise acknowledge that they're there.

They succeed in disturbing you without actually disturbing you, it's great (for them).