I’d prefer similar to Doctorow, that governments simply make scraping and republishing (anything that anyone who consents can access) explicitly legal, then alternate frontends may appear with their own algorithms.

Bingo. This is the path forward and solution to so many ills. IP needs a big rethink for the internet.