The same effect is responsible for an unavoidable flaw with stereo loudspeakers, where you have differing path lengths between your ears and each speaker. Try playing some mono pink noise on stereo speakers and moving your head, then compare with the same sound hard-panned to a single speaker. It's most obvious when you're close to the speakers and in an acoustically dry environment. If you add lots of additional reflections you'll generate many overlapping interference patterns that will average out to a smoother frequency response. This is one reason why adding a real physical center channel can improve clarity of dialogue in movies.

Real hi-fi enthusiasts sit at the reference listening point of their system (the third corner of an equilateral triangle placed on the speakers). Everyone else won't notice the difference; they listen to 256kbps Spotify anyways.