"one of the few human characteristics that hasn't been used to justify discriminations"

Of course it has. That notions like “Napoleon complex” circulate in pop culture, suggests that society broadly considers short-statured people to be somehow disadvantaged and, moreover, it can be funny when the short-statured kick against the pricks. Also, the demand that a suitable male partner ought to be over six feet tall, is absolutely widespread, both in match-seeking profiles on online-dating platforms and in dating fora where women give advice to each other. (Or, if you want to reverse the sexes, Truffaut’s gag in Baisers volés about dating a tall girl, wouldn’t work if people broadly didn’t feel that there was something weird about this.)