If there were a significant amount of such people coming in it would push up the cost of living for the average-(or below-average)-income Japanese citizen.

Look at any of the Western cities that became attractive for retirement or "investment properties" for wealthy international folks... it makes everything more expensive for everyone already there.

If the concern was just about preventing things from changing for existing citizens, restrictions across the income band would seem to make much more sense.