Remix.run Logo
wiredfool 2 hours ago

I run some moderate profile gov and ngo opendata sites, and I’d say that bot like traffic is 99% of the requests we’re seeing on some sites.

Mostly current valid user agents, lots of ip addresses, but the traffic patterns are not organic. I’m not clear if it’s bad ai scraping or dos, but at some level it’s indistinguishable.

reconnecting 2 hours ago | parent [-]

If you can email me, I'd be happy to volunteer some help looking into this for your org, as we've made some tool to investigate bots (open-source).