The incentives are alright. Publishers who now start publishing too much low quality slop will lose readers (who has time to read all those low quality publications). Less readers leads to less citations, which will drag dawn their impact factor resulting in less authors willing to pay a high publication fee.

For those fields with an existing market, meaning there is more than one high quality journal, the market will provide the right incentives for those publishers.

I doubt that this is true except maybe for the top journals. Mid and low tier journals cater to scientists whose main incentive is to publish no matter how while moderately optimizing for impact factor (i.e. readers and citations). This lower quality market is huge. The fact that even top tier publishers have created low-ranking journals that address this market segment using APC-based open-access models shows the alignment between publisher and author interests will not necessarily lead to increasing quality, rather the opposite.

For academia's sake I hope you are correct, but my experience of the system leads me to suspect otherwise, though only time will tell.

One hope might be that it incentivises institutions away from the publish or perish mind set and starts to discourage salami slicing and other such practices, allowing researchers to focus on putting out less work of a higher quality, but I suspect the fees would need to be larger to start seeing this sort of change.