Possibly, but not necessarily.

Your argument is that it's just more attractive. That might be true. That would also mean there's an underlying reason for females finding this attractive. This could be due to taller males being able to protect their offspring from other males, hence more of us are descended from taller males and females that found taller males attractive.

But being taller could also give other advantages like perceived or actual dominance over other males which is, in itself, an attractive quality.