It does matter because it's your network that gets you your next job and colleagues remember who was doing a good job and helping meet goals and those who didn't.

We were talking about keeping a job, not finding a new one.