People are rightly judged for saying they're "in the middle" because too often their "middle" is just whatever they perceptually decided the position of the left and right was and then they picked their position in reaction to that, rather then out of any consideration of the issue.

People love to be "in the middle" and thus "reasonable".

This is why the entire exercise of finding a place on a political spectrum is a trap and a scam, the only thing that really matters:

Are you an extremist or a moderate? Because I can get along with a moderate person on the left, right, or anywhere in between. By the same token extremists regardless of stripe are unbearable.

You're both right. We don't distinguish between the reasonable middle grounders and the unreasonable ones. More broadly, we don't distinguish between reasonable and unreasonable arguments. We never have. Truth as determined by humans is basically a popularity contest.