> Until scrapers start implementing it en masse

If it becomes widespread (as it has been doing), they will. Anubis' strategy only works while it remains a niche approach only adopted by a small number of sites.

▲

Macha 3 hours ago | parent | next [-]

And then people will move to a new solution.

I think this is a pragmatist vs idealist debate. The pragmatic answer is that Anubis solves a problem now, and so it will be used until either it doesn't or a better solution presents itself. The idealist approach is that it's obvious that there's ways to get around Anubis, and so some people argue from there that it shouldn't be used. But the only other alternatives being offered are to either eat up the costs (in server resources or engineering time), or to go behind cloudflare with its own tradeoffs.

▲

greyface- 3 hours ago | parent [-]

Anubis is the penicillin of web hosting.

Every individual practitioner has a strong incentive to overuse it, because it's extremely effective for them individually. Every additional practitioner that uses it increases the selection pressure on their collective adversary to develop resistance. Eventually, it reaches a tipping point and becomes ineffective. But the ecosystem-level impact of its historical use remains, and makes things worse for everyone.

Recently, the baseline expectations of the Web have shifted, and I need to enable JavaScript to read a non-CloudFlare-proxied simple HTML site like LKML. I do not expect this shift will revert once Anubis outlives its effectiveness.

▲

Macha 3 hours ago | parent | next [-]

Viruses do not have motivations, so it’s hard to assign blame to them. The companies running badly behaving AI scrapers are run by people with more of a mind than viruses, so please direct your complaints about the second order effects of their actions that way, rather than on their direct victims.

▲

Krutonium 3 hours ago | parent | prev [-]

I do want to note that my website has Anubis, and does not require Javascript. You can configure Anubis to test in other ways, JS is just the default.

	▲	greyface- 3 hours ago \| parent [-]
		Thank you. I decrement my Anubis-ire counter by 1 whenever I encounter a site configured like this.

▲

drum55 3 hours ago | parent | prev | next [-]

The barrier is a two sentence prompt to Claude to add a patch to curl that does 500MH/s to the 50kh/s the javascript version does on the same hardware, it only works because it’s obscure and largely irrelevant.

▲

inigyou 3 hours ago | parent | prev [-]

Then they'll change what Anubis does.