Presumably whoever is doing the scraping would need to pay, to get rate limits conducive to scraping.

Making requests is inherently cheaper than delivering responses, even with caches. Efficiency improvements can buy a little time on a given resources but won't solve the problem of bot saturation now that everybody can spawn a custom bot in about 12 seconds and is being encouraged to do so.

Rate limits, blocking, and pay-per-use are the only roads out and even those might not last as models get better at hacking and masquerading.

The internet we want to use LLM's with is simply not one that can support LLM's, and with LLM's not going anywhere, the whole experience of the internet is going to be forced into some radically less open and more expensive paradigm.

Policies like this just represent the beginning of the transition.

> We need a new non-commercial version of the internet.

> you pay to access social media optimized to be interesting...

So is it commercial or non commercial?

Nothing is stopping you from creating a social network that is pay gated. Go build it. If you can't get anyone to sign up perhaps you'll realize it's not so easy as scapegoating addictive social media.

There is a whole cottage industry of people who legitimately make their living criticizing Facebook. It's a consumer software product. Yet few of these people seem to have their conviction extend to building an alternative that ever catches an audience. Why is that? Because addiction? Any other excuses?

> We need a new non-commercial version of the internet.

Except it's non-commercial, therefore valuable, therefore commercialized, therefore commercial.

You'd need a force strong enough to prevent it from falling prey to this tragedy of the commons, and that force would need to be stronger than the incentives to commercialize it. And that's where plenty of contemporary scraping-based salaries lay.

That's this internet. Nothing precludes you from dropping big tech and solely engaging with the "indie web"[0][1] if you so choose. You're free to build your own website with its own RSS feed, join webrings[2] of like-minded people, and engage organically.

[0] One of many similar initiatives, I'm sure. Not an endorsement.

[1] https://indieweb.org/

[2] https://en.wikipedia.org/wiki/Webring

Careful what you wish for. The ticket to entry will (have to) be a proof of identity, the ultimate nail in the coffin of privacy and anonymity on the web.Outside of that, chaos.

Impossible to create

AT Protocol and PDS’. BitTorrent to share and distribute bundles of data.