> That means there’s an invisible hand keeping prices up, or basically that the market is not free enough.

Nowhere in the economic theory there is a proposition stating that prices should fall below affordable levels, given enough competition.

We are not talking about economic theory. We are talking about house prices. Time after time it has been seen that free-enough market can lower the prices to affordable levels.

We don't need economic theory for that it's just common sense. Humanity has been erecting structures to live in for approximately our entire existence. The modern economy is mechanized. How could a wood frame structure or a small high rise possibly be unaffordable if the market is functional?

Stop and think for a second. Someone in good health with a willingness to DIY and a sufficiently flexible schedule can literally build their own house from the ground up. It's a substantial time investment but not actually as much as you might think. Housing isn't very resource intensive compared to the rest of the modern economy.

The only possibilities I can imagine to explain unaffordable housing are broken regulations, critical levels of resource exhaustion, natural or man made disaster, and gross economic dysfunction.