| ▲ | harshreality 4 hours ago | |||||||
It is not absolutely useless, empirically, which you'd discover if you had a website getting hammered by bots and experimented with anubis as a countermeasure. While dedicated scrapers/attackers could work around it, and they could do so much more efficiently than the client-side js, almost none of them do. Unless you like paying additional hosting resource fees to serve bots, it's a worthwhile option, and is less annoying to typical human visitors than cloudflare's interactive captcha/challenge which is what most people use. The main author is aware that the algorithm is far from ideal for this purpose. See https://news.ycombinator.com/item?id=48869064 . If more bots start to answer the primitive challenge anubis uses now, that'll hasten implementation of a different algorithm. Don't let the perfect be the enemy of the good enough. For now, the algorithm or challenge scheme almost doesn't matter. Since it's much smaller-scale than cloudflare's challenges, that's probably why very few scrapers and botnets bother to solve anubis's trivial sha2 pow. Targeted attacks may not be repelled at all. That's not the point. | ||||||||
| ▲ | kro 4 hours ago | parent [-] | |||||||
It does not even require the PoW thing Anubis does. I've setup a simple logic that just: Checks for existence of a specific static cookie, if it does not exist, output a small page that sets the cookie via JS and reloads. Sadly this kills Noscript, but it would be possible to add a form in <noscript> that when submitted sets the cookie serverside. Is this trivial to bypass? Yes. It still keeps out 95% of unwanted bots. Reality is most do not target you specifically they just want to mass-scrape with low effort. Running headless browsers is way more expensive for their op I've extended this with a FCRDNS checked exclusion for Googlebot. Another quite effective measure I figured out was checking the existence of Sec-Fetch-Dest header if the User-Agent claims to be a modern browser. If you don't want to close down too much. Also, I only apply these rules to routes that are not cheap and cached. | ||||||||
| ||||||||