There is a problem with the number of dimensions. Even a t-shirt is described best in neck, chest, waist, and you could add several parameters for sleeves and also for heights. Neither consumers nor manufacturers can really handle the combinatorial explosion, so you have to boil it down to one or maybe two dimensions.

But the reason even that isn't done is mostly history and market expectations. There are clothing categories that sell in actual dimensions, and (aside from the terrible dimensional accuracy of clothing in general) it works fine. But those are all on the "men's" side, and it seems the industry believes women will not like buying based on actual numbers.