If you don't publish content to the public web anymore, you don't have to worry traffic or scraping or bots

Maybe it'll just be cheaper for CDNs or whatever to sell the data they serve directly instead of doing extra steps with scraping

The only answer is WebDRM.

It's easy to pretend you're human, it's hard to pretend that you have a valid cryptographic signature for Google which attests that your hardware is Google-approved.

Crawling is the price we pay for the web's openness.