Why would any scraper choose to pay you? They have previously been able to scrape your content for free and can continue to scrape your content for free.

The problem was never the lack of a way to pay you, they just don't care if the transaction is fair.

▲

hephaes7us 15 hours ago | parent | next [-]

Presumably, if you pay and respect rate limits, you get to bypass the anti-scraping measures?

Admittedly, the price that most will be willing to pay is indeed capped by the cost of the alternatives (bypassing anti-scraping measures via proxies, etc). Still, that means that there's a nonzero rational price.

	▲	cvadict 14 hours ago \| parent \| next [-]
		> Presumably, if you pay and respect rate limits, you get to bypass the anti-scraping measures? Far cheaper to just use your hyper-intelligence machines to bypass the anti-scraping measures than to pay everyone. As of today you can only block the scattershot scrapers trying to slurp up everything at minimal effort/cost. Thankfully those make up the vast majority of traffic, so even dead-simple measures like Anubis block them. Whereas an advanced scraper is getting through any technical measures you can conceivably put in place (including the SOTA paid/cloud offerings). At the high end, the agents just drive regular browsers, mobile device, etc. running through a residential proxy / mobile CGNAT'd ip, solving captcha's in real-time, reasoning about the protection measures in place + adapting in real-time, etc. etc. They are virtually indistinguishable from humans. If you want humans to be able to see your content, the bots will too, as your average bot can now be far smarter + faster + adaptable than your average human.
	▲	warkdarrior 13 hours ago \| parent \| prev [-]
		> Presumably, if you pay and respect rate limits, you get to bypass the anti-scraping measures? Payment are not made for "presumably". The website should offer a clear contract: "to turn off anti-scraping measures for X amount of time, pay $Y".

▲

lowbloodsugar 12 hours ago | parent | prev [-]

Seems like we need to make this the default for everything - including humans.