When the environment changes and we convert forests to farms or urban centers, wildlife of course disappears. Then over time the surviving creatures will evolve and adapt to become suited to living around these urban centers.

When you remove these urban centers the same thing happens. You are changing the environment. So of course there is an initial culling as the selection process activates.

We tend to think of depopulating actions in a single unidirectional way. Nature -> Urban = Bad and depopulating. The reality is that nature is neutral. It's Not unidirectional and nothing is bad or good. Believe it or not I'm sure species do go extinct from the conversion from urban to more wild environments. It happens both ways and again, extinction isn't inherently bad either.