Not what you want to hear, but genes, probably ...

Yeah I think that is the biggest factor. I got family members that are built like brick shithouse from a life since childhood of physical labor that still have their hair into their 70s, there is no way they weren't maintaining high levels of testosterone their entire life but it didn't seem to matter.

Probably. Money seems to help too, though. A few Hollywood actors have miraculously reversed their male pattern baldness.