That's an entirely valid point of view, and one plenty of people share. Being accountable for other people's mistakes sucks.

The only issue with it is that you can only get positive side for work you do directly, which limits the scale of what you're seen to be capable of doing. I lead a team of about 50 (7 directly, plus their teams), so I get to lead on quite big projects that I'd only work on smaller parts of I wasn't a manager. I enjoy that aspect of my role. But yeah, taking the blame when someone screw up isn't fun.