| ▲ | gruez 3 hours ago |
| >If scrapers reuse a cookie, traditional anti-abuse mechanisms will work. Scrapers can't change their strategy? It's not hard to think of mitigations: keep on using a session until you're banned, and failing that rotate your IP. Nowadays you don't even need to think. You can probably tell a LLM "write me a scraper that has adaptive anti-anti-bot mechanism". |
|
| ▲ | embedding-shape 3 hours ago | parent | next [-] |
| > Scrapers can't change their strategy? Yes, but you normally don't try to fight all strategies up front, you add stuff on a need-to basis. Currently, lots of people are in the stage of "need to get this thing using bunch of IPs to stop hammering my servers", apparently Anubis helps them with that. If that stops working, other tactics are tried, and so it goes on for the life of your service. Hasn't stopped yet, doesn't seem to be stopped soon either. |
| |
| ▲ | gruez 3 hours ago | parent [-] | | That argument also works against anubis. Why bother forcing clients to solve a PoW challenge, when you're fighting a bunch of unsophisticated attackers that will be deterred by a javascript challenge that requires them to solve `result = x + y`? | | |
| ▲ | embedding-shape 2 hours ago | parent | next [-] | | Point is, you get problems because of something like this, you try either approach. And when that stops working (because eventually it will), you add something else. Some work better against some actors, but also feels like it never ends, so slowly you add more things. I don't think Anubis is uniquely able to stop that particular type of annoyance, it's just one of the tools you can see if it helps. If you want to try something else before, makes a lot of sense to do so. | |
| ▲ | xena 2 hours ago | parent | prev | next [-] | | This is why there's the meta refresh challenge as an option: https://anubis.techaro.lol/docs/admin/configuration/challeng... Turns out that in practice there's some bots that implement support for this but not JavaScript. | |
| ▲ | mitxela 2 hours ago | parent | prev [-] | | Caution: Techaro is very hostile to this line of thinking. He believes the PoW is essential. And that's fine - different projects use different approaches, like go-away and dogwaf. |
|
|
|
| ▲ | bulder 3 hours ago | parent | prev [-] |
| Plenty of these abusive scrapers are utilizing retail residential proxies, which will be applying forced rotations to avoid "burning" their compromised and or otherwise surreptitiously utilized IP address. |