> Many sites only allow Google scrapers in robots.txt and the public will fight you hard if you scrape them without permission.

This just lets a monopoly replace the website instead of distributing power and fostering open source. The same monopoly that was already bleeding off the web's utility and taxing it.