We carved out women’s sports because otherwise there would be no biological women in competitive sports, and that was considered to be a significant enough exclusion of half the human population as to warrant such direct intercession.

Whether or not a similar case can be made for other categories does not have bearing on the case for sex categorization. Such claims can and should stand on their own merits.