Remix.run Logo
lxgr 4 hours ago

It's absurd how unusable Cloudflare is making the web when using a browser or IP address they consider "suspicious". I've lately been drowning in captchas for the crime of using Firefox. All in the interest of "bot protection", of course.

lucasfin000 4 hours ago | parent | next [-]

The real frustrating part is that Cloudflare's "definition" of suspicious keeps changing and expanding. VPN users, privacy-first browsers, uncommon IP ranges, they all get flagged. The people most likely to get caught by these systems are exactly the ones who care most about their privacy, and not the bots that they are apparently targeting.

gruez 4 hours ago | parent | next [-]

>The real frustrating part is that Cloudflare's "definition" of suspicious keeps changing and expanding.

That's... exactly expected? It's a cat and mouse game. People running botnets or AI scrapers aren't diligently setting the evil bit on their packets.

jagged-chisel 3 hours ago | parent | next [-]

That’s obviously because they’re not being “evil”

lxgr 3 hours ago | parent | prev [-]

So the stable state here is all humans eventually being locked out? (Bots are getting better every day; I doubt the same is true for all humans, including those with weird browsers or networks unwilling to install some dystopian Cloudflare "Internet passport".)

But hey, at least some bots are also not making it past Cloudflare!

WatchDog an hour ago | parent | next [-]

The inevitability is that these kinds of services just won't be offered without identifying yourself.

Claude's free tier requires a phone number just to try it.

small_scombrus an hour ago | parent | prev [-]

> So the stable state here is all humans eventually being locked out?

Yep. The most easy to implement stable state for any system where you're aiming to prevent misuse is to just prevent use

Aurornis 37 minutes ago | parent | prev | next [-]

> The people most likely to get caught by these systems are exactly the ones who care most about their privacy, and not the bots that they are apparently targeting.

In my brief experience with abuse mitigation, connections coming from VPNs or unusual IP ranges were very significantly more likely to be associated with abuse.

It depends on your users. VPNs aren’t common at all, even though you hear about them a lot on Hacker News. For types of social sites where people got banned for abuse (forums) the first step to getting back on the forum was always to sign up for a VPN and try to reconnect. It got so bad that almost every new account connecting via VPN would reveal itself as a spammer, a banned member trying to return, or someone trying to sock puppet alternate accounts for some reason.

The worst offenders are Tor IP addresses. Anyone connecting from Tor was basically guaranteed to have bad intentions.

I heard from someone who dealt with a lot of e-mail abuse that the death threats, extortion, and other serious abuse almost always came from Protonmail or one of the other privacy-first providers that I can’t remember right now. He half-jokingly said they could likely block Protonmail entirely without impacting any real users.

It’s tough for people who want these things for privacy, but the sad reality is that these same privacy protections are favored by people who are trying to abuse services.

whatisthiseven 4 hours ago | parent | prev [-]

Which VPNs are people using that actually care about the user's privacy? Most of them don't, sell their home IP to buyers, sell their DNS history to others, etc. Worse, some of them could require invasive MITM cert stuff most users will just click yes through.

I have yet to see a use case for VPNs for the casual internet audience, and for a tech savvy user, their better off renting through some datacenter or something, which at that point is hardly a VPN and more home IP obfuscation. All the same downsides, and at least you get real privacy.

traceroute66 3 hours ago | parent | next [-]

> Which VPNs are people using that actually care about the user's privacy?

Mullvad.

It has been proven in a court of law that when Mullvad says "no logging", they mean it.

They also regularly have security audits and publish the results[2][3]

[1]https://mullvad.net/en/blog/mullvad-vpn-was-subject-to-a-sea... [2]https://mullvad.net/en/blog/new-security-audit-of-account-an... [3]https://mullvad.net/en/blog/successful-security-assessment-o...

monista an hour ago | parent | next [-]

I don't use the VPN, but I still happily use their privacy-oriented (Firefox-based) Mullvad browser.

https://github.com/mullvad/mullvad-browser/

thisisnow 3 hours ago | parent | prev [-]

Second for Mullvad, I am quite distrusting in general but more I know about Mullvad, more I am convinced they really are serious about user privacy

evilduck 4 hours ago | parent | prev | next [-]

Using any popular datacenter's IP range for a personal VPN is likely to be outright blocked.

Imustaskforhelp 3 hours ago | parent [-]

Also you only get 1 IP so its not really anonymous and you definitely would have a fingerprint.

thisisnow 3 hours ago | parent [-]

you just rotate it?

lxgr 3 hours ago | parent | prev | next [-]

I'm forced to use a VPN to occasionally check my US bank account, since a foreign IP address is obviously a harbinger of unspeakable evil (while the friendly Youtube advertised neighborhood VPN is obviously evidence of pure intentions).

gruez 4 hours ago | parent | prev | next [-]

>Most of them don't, sell their home IP to buyers, sell their DNS history to others, etc. Worse, some of them could require invasive MITM cert stuff most users will just click yes through.

Source? I haven't seen any evidence that the major paid VPN providers engage in any of those things. At best it's vague implications something shady is happening because one of the key people was previously at [shady organization].

Imustaskforhelp 3 hours ago | parent | prev [-]

ProtonVPN with bitcoin which you get from a monero swap is a good idea for complete privacy if you want port forwarding.

MullvadVPN is also another great one.

I have heard some good things about AirVPN, but I can absolutely attest for mullvad and to a degree ProtonVPN (Just with Proton, depending upon your threat model, do make the necessary precautions like buying with monero for example)

There are others, but mostly its the 2-3 that I trust.

4 hours ago | parent | prev | next [-]
[deleted]
danielheath 4 hours ago | parent | prev | next [-]

Maybe check your network isn't sending web traffic you're not aware of?

I'm running firefox and seeing the normal amount.

jychang 4 hours ago | parent | next [-]

Most people are on a CGNAT these days, drowning in captchas is the new normal. You’re at the mercy of one of your neighbors not hosting a botnet from their home computer.

perching_aix 4 hours ago | parent | next [-]

For better or for worse, CF's fingerprinting and traffic filtering is a lot more in-depth than just IP trend analysis. Kind of by necessity, exactly because of what you mention. So I'd think that's not as big a worry per se.

lxgr 3 hours ago | parent [-]

Yet here I am drowning in captchas every once in a while, so it's quite a big worry for me.

Maybe I just have to disable all ad blockers and Safari tracking prevention? Or I guess I could send a link to a scan of my photo ID in a custom request header like X-Please-Cloudflare-May-I-Use-Your-Open-Web?

perching_aix 3 hours ago | parent [-]

> Yet here I am drowning in captchas every once in a while, so it's quite a big worry for me.

I think I was sufficiently clear that I was specifically talking about CGNAT-caused IP address tainting being an unreasonably emphasized worry, not the worry about their detections overall misfiring. Though I certainly don't hear much about people having issues with it (but then anecdotes are anecdotal).

> Or I guess I could send a link to a scan of my photo ID in a custom request header like X-Please-Cloudflare-May-I-Use-Your-Open-Web?

Sounds good, have you tried?

Not sure what's the point of these comically asinine rhetoricals.

tokioyoyo 4 hours ago | parent | prev [-]

Not even remotely true, I genuinely have no idea what you're talking about. The only time I get captcha'ed is when I sometimes VPN around, or do some custom browser stuff and etc. I'll even say I get captcha'ed less now than maybe 5 years ago.

cogman10 4 hours ago | parent | prev | next [-]

Every so often, usually after a firefox update, CF will get into a "I'm convinced your a bot" mode with me. I can get out of it by solving 20 CAPTCHAs.

hansvm 4 hours ago | parent [-]

It's probably just a higher rate of autonomous vehicles needing stop signs and buses identified at that moment, and cognitive bias causes you to only remember when that happens when you recently performed an update. /s

cogman10 4 hours ago | parent | next [-]

My assumption is that CF has something like a SVM that it's feeding a bunch of datapoints into for bot detection. Go over some threshold and you end up in the CAPTCHA jail.

I'm certain the User-Agent is part of it. I know that for certain because a very reliable way I can trigger the CF stuff is this plugin with the wrong browser selected [1].

[1] https://addons.mozilla.org/en-US/firefox/addon/uaswitcher/

gruez 4 hours ago | parent | prev [-]

>It's probably just a higher rate of autonomous vehicles needing stop signs and buses identified at that moment

I can't tell whether you're serious but in case you are, this theory immediately falls apart when you realize waymo operates at night but there aren't any night photos.

hansvm 4 hours ago | parent [-]

Thanks for the comment. Lack of seriousness is now appropriately indicated.

g-b-r 4 hours ago | parent | prev [-]

Maybe you allow tracking and cookies?

Eji1700 4 hours ago | parent [-]

I don't, and I rarely have issues with firefox. Private + blockers + VPN causes, expected, issues but otherwise i'm usually fine?

girvo 3 hours ago | parent | prev | next [-]

Surprising really, because I'm a Firefox + Ublock Origin die hard and I never get Cloudflare captchas. Wonder what the difference is? I have CGNAT turned off, if that matters at all (probably not).

lxgr 3 hours ago | parent [-]

I could definitely imagine a public IPv4 with lots of good, logged-in Cloudflare traffic to act as a positive signal for their heuristics, possibly even overriding the Firefox penalty.

ehnto 4 hours ago | parent | prev | next [-]

I recently had the insane experience of filling out 15 consecutive captchas, after, I had checked out and entered my payment information into the payment processor widget. I just wanted to submit the order. I was logged in to their website, and the bank even needed a one time code for payment. If the bank is pretty sure I am human then your ecomm site can figure it out surely.

lxgr 3 hours ago | parent | next [-]

That's my favorite combination: Shitty bot detection meeting shitty payment security systems.

At least outside the US, there's 3DS as an (admittedly often high friction) high quality cardholder verification method, but in the US, that's of course considered much too consumer-hostile, so "select 87 overpasses" it is.

amatecha 3 hours ago | parent | prev [-]

A while back I was buying tickets for a gondola for a trip in Europe and the checkout process failed during payment because their site didn't load their analytics/tracking stuff with proper error-handling, so when my ad-blocker prevented the tracking stuff, their checkout process failed to handle my CC's 2-factor auth and the checkout would fail. Had to contact my CC company and work with the gondola company to tell them what they're doing wrong so they could fix their website code. Pretty sad to know whoever built their stuff actually shipped a checkout flow (for a VERY popular tourist destination) without testing with ad-blockers enabled.

lxgr 3 hours ago | parent [-]

To be fair, this sometimes seems on the ad blocker. I've definitely seen mine accidentally nuke part of the payment Javascript (or maybe the 3DS iframe?) because some substring of it matched some common ad URL, which is obviously unrecoverable for the site itself.

binaryturtle 3 hours ago | parent | prev | next [-]

I'm with a slightly older Firefox and can't use many websites at all anymore because the Cloudflare cancer.

Of course then you got sites like gnu.org too that block you because your slightly outdated user agent.

segmondy an hour ago | parent | prev | next [-]

trying using firefox and then using a cellphone network for internet. sometimes i can't access a site, because i get infinite captcha. i know what a damn bus, stairwell, stop light or motorcycle looks like.

onion2k 4 hours ago | parent | prev | next [-]

Is that because botnets spoof being Firefox? It's not really fair to blame Cloudflare it is. That's on the bots.

doctaj 4 hours ago | parent | next [-]

In what way would that not be fair? Their product giving false positives (unnecessary challenges for a normal browser humans commonly use) to real people is definitely their fault.

gruez 4 hours ago | parent [-]

>Their product giving false positives (unnecessary challenges for a normal browser humans commonly use) to real people is definitely their fault.

Is it TSA's "fault" that non-terrorists are subject to screening?

lxgr 3 hours ago | parent | next [-]

No, but it's entirely within TSA's hands to make that process as frictionless as possible.

(It's a different question whether zero friction is actually desired, or whether some security theater is actually part of the service being provided, but that's a different question.)

forkerenok 4 hours ago | parent | prev [-]

We're discussing the quality of screening here, not the act/necessity of screening itself.

gruez 4 hours ago | parent [-]

>We're discussing the quality of screening here

The "quality" of TSA's screening seems be pretty bad too given how many people have to go through secondary screening vs how many terrorist they catch (0?)

bdangubic 3 hours ago | parent | next [-]

they caught 11 million by now (just as arbitrary as your 0 but probably more accurate since we haven’t had a large terrorist attack since they got the gig to serve and protect and before we lost thousands of lives…)

gruez 2 hours ago | parent [-]

>they caught 11 million by now (just as arbitrary as your 0 but probably more accurate

Nice try but I used "caught", not "stopped", which requires they actually apprehended someone, not just prevented some hypothetical attack.

>since they got the gig to serve and protect and before we lost thousands of lives…)

You could easily reuse this argument for cloudflare: "if it wasn't for such invasive browser fingerprinting openai would be drowning in bajillion req/s from bots."

bdangubic an hour ago | parent [-]

> “if it wasn't for such invasive browser fingerprinting openai would be drowning in bajillion req/s from bots."

of course they would be drowning! I have no issues with what CF is doing. too funny that people use tools like chatgpt and expect privacy?!

DonHopkins 4 hours ago | parent | prev [-]

They are failing to meet there quotas of shooting innocent people in the face, so ICE is helping out.

lxgr 3 hours ago | parent | prev | next [-]

No, using a stupid authentication/verification method with lots of false positives is always on whoever deploys it.

Imagine an apartment building with a flimsy front door lock that breaks all the time, and the landlord only telling you that that can't be helped because of all the burglars.

josephcsible 3 hours ago | parent | prev | next [-]

If it's just as easy to spoof being Chrome as it is to spoof being Firefox, then it is indeed fair to blame Cloudflare if they give Firefox users more CAPTCHAs than Chrome users.

conradkay 4 hours ago | parent | prev [-]

Not really, there's camoufox but the vast majority use modified chrome/chromium

dawnerd 3 hours ago | parent | prev | next [-]

I’ve been getting it in safari too. It’s ridiculous frankly. My residential ip must have been flagged or something. The part that’s really annoying is its trivial for bots to bypass.

lxgr 3 hours ago | parent [-]

> I’ve been getting it in safari too.

I'm getting it on iCloud Private Relay all the time. It honestly makes it kind of useless.

Maybe that's the point? But then again, doesn't Cloudflare run part of it!? And wasn't there some "privacy-preserving captcha replacement" that iOS devices should already be opting me in to? So many questions, nobody there to answer them, because they can get away with it.

> The part that’s really annoying is its trivial for bots to bypass.

Not the ethical bots, though! My GPT-backed Openclaw staunchly refuses to go anywhere near a "I'm not a robot" button.

tshaddox 3 hours ago | parent | prev | next [-]

Is anyone talking about the fact that this is a fundamental design flaw of the web? Or arguably even the entire Internet?

3form 2 hours ago | parent [-]

It's hard to call something a "fundamental flaw of web" if it wasn't an issue for 30 years. Unless you mean something more general that I'm missing.

amatecha 3 hours ago | parent | prev | next [-]

These days I just close sites that show that "checking if you're a bot" shit. If this is how the web is going to be now, I don't care, I'll just not use it. I didn't need to see that article or post that badly anyways. I'm tired of paying the price for the sociopathic, greedy actions of others. It's especially bad for anyone who uses an open source OS like Linux or *BSD (to the extent many sites just block me automatically with a 403 Forbidden simply for using OpenBSD + Firefox, completely free pass if I try the same site from a Windows or Linux computer).

jgalt212 3 hours ago | parent [-]

We use Cloudflare to protect our content, but at the same time our machines mostly run Linux / Firefox so it really is quite a frustrating relationship. It really bums me out how much of Turnstile boils down to these two questions:

is it Linux (or similar)?

is it Firefox?

If yes, to one or both, you're blocked! Clearly millions of dollars of engineering talent and petabytes of data collection should be able to come up with something more nuanced than this.

dheera 4 hours ago | parent | prev | next [-]

Exactly. For the most part all this bot protection is only protecting these websites against humans.

I don't do free work. I'm not going to label 50 images of crosswalks and motorcycles for free.

ronbenton 4 hours ago | parent [-]

> For the most part all this bot protection is only protecting these websites against humans.

Curious how do you know this?

EGreg 4 hours ago | parent | prev [-]

Well, that's for the public internet.

I'm building Safebox and Safecloud, where this won't be the case anymore. Not only will you have a decentralized hosting network that can sideload resources (e.g. via a browser extension that looks at your "integrity" attribute on websites) but also the websites will require you to be logged in with a HMAC-signed session ID (which means they don't need to do any I/O to reject your requests, and can do so quickly)... so the whole thing comes down to having a logged in account.

https://github.com/Safebots/Safecloud

As far as server-to-server requests, they'll be coming from a growing network of cryptographically attested TPMs (Nitro in AWS, also available in GCP, IBM, Azure, Oracle etc.) so they'll just reject based on attestations also.

In short... the cryptographically attested web of trust will mean you won't need cloudflare. What you will need, however, to prevent sybil attacks, is age verification of accounts (e.g. Telegram ID is a proxy for that if you use Telegram for authentication).

password4321 4 hours ago | parent | next [-]

Wow, if Seinfeld can have a soup nazi, I think it's within reason for you to be called the internet nazi.

"No s̶o̶u̶p̶ internet for you!"

Good luck!

ale42 4 hours ago | parent | prev [-]

This was sarcasm, right?