Exactly the same thing happens with in-office work. I've certainly had that experience! But it's hard to talk about because in-person work is the "default", and it feels like if you have problems with that it's a sign of problems with you.

(Not to mention that, in both contexts, it's very hard to pull apart being remote vs in-office from other aspects of the work and culture.)