Remix.run Logo
Jolter a day ago

It’s effective against teenagers maybe. Not so much against Amazon, Meta or wherever botnet/crawler is coming out of China these days from up-and-coming AI companies.

lucb1e 2 hours ago | parent | next [-]

Then block all of Amazon, Meta, or wherever botnet/crawling traffic is coming from that doesn't honor robots.txt, sends DDoS reflection traffic, submits SMTP messages (in large volumes, not just probing) for domains they're not authorized for with SPF, or whatever else applies to the protocol you're using

If they can't keep their ranges clean to a reasonable degree, their customers will need to move if they want to access your part of the internet. New sign-ups will always be hard, so some amount of abuse is expected, but if it's the same abuse traffic for weeks after you've notified them, well, it stops being your problem at some point

Jolter 2 hours ago | parent [-]

See the other comments in this thread. The perpetrators are unknown and are jumping between residential IPs. Possibly botnets?

lucb1e 2 hours ago | parent [-]

Then see my other replies in the thread where I've specifically addressed residential IPs, e.g.: https://news.ycombinator.com/item?id=48163060

tonetegeatinst a day ago | parent | prev [-]

I mean you could block entire AS numbers that relate to amazon or big tech datacenters

tangledhelix 21 hours ago | parent [-]

wouldn't help, much of the traffic we've observed look closer to ddos patterns - IPs from all over the world, many different networks, each IP makes one request only, doesn't come back. highly distributed, no form of blocking would be effective except maybe captcha or proof of work.