>But even if that was true, who's to say that population has to keep growing?

The issue is bigger then that. It's weird though. First for context, when population grows, that growth is compounding usually. There's an acceleration.

Now that the population is in freefall, the freefall is also compounding. Like there are structural impacts that will effect most of the asian countries (where this problem will hit first) that will effect them economically.