I am responsible for my colleague's careers, and they for mine.

This guy is apparently a jerk, who apparently thrived in a jerk atmosphere. Don't listen to him.