It's not just being able to afford it. It's also how easy can you sell it. Basically, how liquid is the house, and how many expenses are there on the transaction, impacts how much it's worth to buy.

My thought experiment is basically: If I get a new job in a new city, is it smart buy a house right away? And if the answer is not always "yes", then obviously buying is not always the best option.