I'm the same way. Removing glasses is an excellent hack, but it also means I'm not making eye contact as I'm just looking at the person's general direction.

Earlier in my career during interviews I'd always tell the interviewer "I'm listening, keep talking, I just need to look at the floor or wall to concentrate or I get too distracted by your animated-ness" and this was in the days before autism entered the zeitgeist. Nobody cared or thought it was particularly odd as far as I could tell. We were accustomed to nerds who sat alone thinking at computers most of the time being quirky, and appreciated the need to concentrate (private offices with doors that close!).

But as I get older it's become more necessary to look at something else while listening to anyone tell me anything I intend to grok. And I'm pretty certain ~everyone now assumes I'm autistic because of it. Never been diagnosed as such.

My interpretation and internal modeling of things is highly visual and genuine visual input is a sort of noise that interferes with my mental reconstruction of what I'm hearing. I have no idea if that overlaps with autism.