People absolutely make that argument about property taxes. That's where deferrals or abatements for e.g. elderly or low-income homeowners, or caps on property tax increases come from. Someone may own a home that property taxes price them out of, forcing them to leave their community because they can't actually conjure money from a higher priced home.

I think a lot of tax authorities also don't really aggressively reassess that regularly without a sale, so it also kind of ends up baked in that if you didn't pay that much for the property, it's only theoretically worth that much.

Predatory property taxes were a part of how so many black American farmers lost their land: https://archive.is/exkhR

This is all conceding the argument already. Many of us would happily accept these sorts of limitations on a wealth tax if it means there is a wealth tax.