| ▲ | JohnMakin 20 hours ago |
| Cloudflare offers bot mitigation for free, and pretty generous WAF rules that makes mitigations like this seem a little overblown to me |
|
| ▲ | nospice 12 hours ago | parent | next [-] |
| I'm on the free tier, but I also watch my logs. The vast majority of the traffic I'm getting are scrapers and vulnerability scanners, a lot of them coming through residential proxies and other "laundered" egress points. I honestly don't think that Cloudflare is on top of the problem at all. They claim to be blocking abuse, but in my experience, most of the badness gets through. |
| |
| ▲ | cakealert 9 hours ago | parent [-] | | when you combine a residential proxy with a tool like curl-impersonate (there are libraries in Go for this type of fingerprint spoofing now) they dont even show up as scrapers anymore, just users. especially when they adjust timings to mimic humans. clouflare only blocks the most dumb of bots, there are still a lot of them. this is why cloudflare will issue javascript challenges to you even when you are using google chrome with a VPN, they are desperate to appear to be doing something. and every VPN is used to crawl as well. a slightly more sophisticated bot passes the cloudflare javascript challenge as well, there really is nothing they can do to win here. i know some teams that got annoyed with residential proxies (they are usually sold as socks5 but can be buggy and low bandwidth) so they invested into defeating the cloudflare javascript challenge and now crawl using 1000's of VPN endpoints at over 100 Gbit/s. | | |
| ▲ | oidar 3 hours ago | parent [-] | | Is "residential proxy" another name for an hacked/owned computer that the bots have access to? Or are there legitimate services that sell access to residential IPs? | | |
| ▲ | nospice an hour ago | parent [-] | | People legitimately sell egress. It's "free" money. But of course, if you have a botnet, you can sell that through the same channels, no one is looking too closely. |
|
|
|
|
| ▲ | n1xis10t 20 hours ago | parent | prev | next [-] |
| You can’t deny that it’s fun though. Personally I generally feel like more people should be coming up with creative (if not entirely necessary) solutions to problems. |
|
| ▲ | conception 20 hours ago | parent | prev | next [-] |
| For “free”. |
| |
| ▲ | n1xis10t 20 hours ago | parent [-] | | Did you put “free” in quotes because you need to have paid for stuff from cloudflare to use the “free” thing? If so, I suppose it’s like those magazines that say ”free cd”. | | |
|
|
| ▲ | ATechGuy 17 hours ago | parent | prev [-] |
| It is really free? Genuinely asking. |
| |
| ▲ | gilrain 16 hours ago | parent [-] | | Yes. They upsell more complete solutions, but the free tier is pretty generous. |
|
