If you have a second home in manhattan, you are not the middle class _and_ you can afford it.

If the idea of paying taxes affects your poor fragile mind so negatively then there is an easy solution where you can sell said extra home.

No one is being fucked hard by this other than people who are appalled at the thought that they need to contribute to society for the negative externalities they create, like accumulating excess shelter in regions with a dearth of housing capacity.