On the bright side, that opens an opportunity for 10,000 companies whose only activity is scraping 10MB worth of garbage and providing a sane API for it.

Luckily all that is becoming a non-issue, as most content on these websites isn't worth scraping anymore.

▲

judge2020 3 days ago | parent [-]

*and whose only customers are using it for AI training

	▲	TeMPOraL 3 days ago \| parent [-]
		They can afford it because the market rightfully bets on such trained models being more useful than upstream sources. In fact, at this point in time (it won't last), one of the most useful applications of LLMs is to have them deal with all the user-hostile crap that's bulk of the web today, so you don't have to suffer through it yourself. It's also the easiest way to get any kind of software interoperability at the moment (this will definitely not last long).