That could be an advantage if your work requires some kind of sustained concentration, for the other party at least.

I like using by headphones (which are big and over the ears) as a way to signal when I’m on concentration mode and don’t want to talk, but I do that maybe 30-40% of the time.