Remix clone Hacker News

new | show | ask | jobs Github

pryelluw 4 days ago

Who gets to decide what is a good bot?

pythonaut_16 4 days ago | parent | next [-]

Presumably the site owner with Cloudflare providing enforcement.

Generally isn't a good bot one that respects robots.txt and is respectful of the site's resources by not being spammy?

wnevets 4 days ago | parent | prev | next [-]

by checking which follow robots.txt?

wmf 4 days ago | parent | prev | next [-]

According to their plan, the good bots pay for scraping.

overfeed 4 days ago | parent | prev | next [-]

> Who gets to decide what is a good bot?

Paying Cloudflare's tolls => good bot.

nullpoint420 4 days ago | parent | prev | next [-]

Cloudflare, apparently.

mark212 4 days ago | parent [-]

no, their customers. Why do you assume that people who run websites are clueless and lack agency in this?

	▲	nullpoint420 4 days ago \| parent [-]
		inb4 "we're updating our Terms of Service - all Cloudflare-based scrapers will now automatically be enrolled to scrape your cached websites" and they'll explain how it's great because it's only reading the cached versions, not hitting upstream.

tccole 4 days ago | parent | prev [-]

Me… obviously