The idea that people object to the removal of old structures because they're more attractive is easy to disprove. People have protested against the removal of structures that even they admit are ugly.

There have been protests against removing rusting oil storage tanks. Our "industrial heritage" apparently.

People just grow to accept the eyesores and start to think of them as part of the local identity. You can find complaints about how ugly the Eiffel tower was when it was built. Symbol of the city now.

Some people opposed removal of ugly structures because they had other significance does not prove no people opposed construction of ugly structures because they were ugly.