Because if you’re the manager, the time of your team is more important than your time is

Excellent. I'll stop hiring and kill the roadmap discussion then, since obviously that one morning guy on my team is more important than anything or anybody else.

Get a grip. Most managers try to accomodate folks, but core working hours exist for a reason.