Remix clone Hacker News

new | show | ask | jobs Github

▲

inigyou 2 days ago

It says "accused of"

▲

shakna 2 days ago | parent [-]

... It also has a quote from Anthropic, admitting they backed off their scraper after a robots.txt update.

▲

inigyou a day ago | parent [-]

Nobody who gives a shit about their software actually working obeys robots.txt. most robots.txt block everything except googlebot, which is an insane policy and you don't have to follow it.

	▲	shakna a day ago \| parent [-]
		Which isn't all that relevant a discussion here. Anthropic saying they did so in this case, means that they admit they were the scraper flooding the domain.