Works the same in the reverse case: I'm a native speaker in the office language, but many of my coworkers are not. One of them is impossible for me to tune out, only because of a difficult accent (his grammar etc are actually better than the ones that I can tune out). It feels as if a mental background process for flagging keywords or something like that keeps bailing out with an exception, "sorry consciousness, you take over". I discovered that I can limit the effect with positive framing as a naturally occurring experiment that grants an interesting window into the inner workings of the mind. At least this way, the distraction does not get compounded by annoyance (which also would be quite unfair). I can only imagine how bad it must be when everyone around you causes that kind of escalation to consciousness.