Not really, not unless you're C-suite or your org size is in the thousands. When Google's looking for a VP to run a 100 person department, they care about your experience running similarly sized orgs as much as they care about your ability to achieve business results. People make fun of empire building but it's absolutely rational on the individual level.