Yes, this is a real problem. Also, in high-demand areas, building more will lower prices compared to the counterfactual but that doesn't mean prices will actually go down. This makes it a harder sell.

and in the near term, you reduce supply by kicking the people out of their homes and they need to find somewhere else