Seeing how elevators are akin to vertical subways I think that problem goes both ways.
Definitely had to factor in elevator time for my commute when I worked on the 38th floor.