Remix.run Logo
Havoc 4 days ago

It’s a bit alarming how cloudflare is establishing itself as arbiter of all things bots…both on blocking and allowing.

Doesn’t seem healthy for the internet as a whole

postalcoder 4 days ago | parent | next [-]

Gonna zag here. If you take a step back, cloudflare has been paving the path for pay for crawl. I think it's a noble and ambitious goal.

While I can understand why you would be alarmed, I can point to almost two decades of lamenting on this forum about how we need better ways of rewarding content creators than ads. Well, this is it.

Moreover, these products weren't built in a vacuum. Most threads about Anthropic and OpenAI have complaints about how these companies were built on stolen content. There's a reality we have to face here and we can't have it both ways.

halJordan 4 days ago | parent | next [-]

Is it a better way though? The problem with ads has always been their abusive nature. From unrestrained pop ups and clickjacking in the 90s to today's pervasive surveillance and profiling.

Cloudflare turnstile is a pop up. This product is pervasive surveillance. It having a cf logo doesn't change that or ameliorate the many many abuses that two decades have shown are part and parcel.

RedRocketFlash 4 days ago | parent | prev [-]

What's incredible is that you have businesses paying CloudFlare to stop their content being ingested by AIs (OpenAI, Anthropic, Self-operated scrapers).

And at the same time, they're paying SEO experts to make that same content easier to be ingested by systems (Google and other Search Engines) which use it for their own AI offerings.

Are you going to be able to make your online content available to Google Search but unavailable for Google Gemini?

fragmede 4 days ago | parent [-]

Yes. Google obeys robots.txt. Set yours to disallow Google-Extended and they won't train Gemini on it.

skybrian 4 days ago | parent | prev | next [-]

Yes, but it’s up to their competitors to build competing services.

cryo32 4 days ago | parent | next [-]

I think it's up to their customers not to encourage consolidation.

grim_io 4 days ago | parent | next [-]

Microsoft is the proof that customers do want that, or that they don't have a real choice.

cryo32 4 days ago | parent [-]

I think it's simpler - they don't give a crap.

Until somewhere down the line. Like when half of Spain gets cut off due to an arbitrary block on a consolidated service facade...

Tade0 4 days ago | parent | prev | next [-]

Customers shouldn't have to micromanage every service and product like that.

We have antitrust regulations for such things.

Catloafdev 4 days ago | parent | prev | next [-]

What Cloudflare competitors offer a similar range of services?

owenthejumper 4 days ago | parent | next [-]

Akamai, Fastly, HAProxy, F5, many others? Talking about bots specifically, not about "workers" etc.

cute_boi 3 days ago | parent [-]

Akamai is nightmare, they block many user mercilessly. They are worst than Akamai. I don't see issue with other though.

cryo32 4 days ago | parent | prev [-]

Do you really need Cloudflare's services to run your business?

Catloafdev 4 days ago | parent [-]

Are you genuinely asking "does anybody even need any of their services"?

yjftsjthsd-h 4 days ago | parent | next [-]

You said "a similar range of services". Basic DDoS protection is probably important, and there are other companies doing that. Beyond that, it's less obvious.

cryo32 4 days ago | parent | prev [-]

Yes. I am asking exactly that objective question.

They are advantageous to leverage in certain situations but essential they are not. We're used to, in the technology industry, looking for or creating problems to solve with services we are aware of. Moving back to necessity and need, do we really? Are we being objective? Most of the time, no.

dpoloncsak 4 days ago | parent | next [-]

Theres a few alternatives, but at a minimum yes you probably need their or a competitor's Name Servers and their public DNS. Rolling your own isn't very feasible.

fragmede 4 days ago | parent [-]

DNS hosting is easy if you did want to roll your own, but everyone one does it, especially your registrar. The more sticky bits of Cloudflare are their cloud services, specifically compute (workers) and database and object storage, all wrapped up into a convenient product (Pages). Also their identity and VPN stuff. They've come a long way since just doing DDoS protection/being a CDN.

Catloafdev 4 days ago | parent | prev | next [-]

DDoS protection is pretty essential. I highly encourage you to read through what they offer if you're not familiar.

4 days ago | parent | prev [-]
[deleted]
skybrian 4 days ago | parent | prev [-]

That too, but they need competitors to switch to.

shimman 4 days ago | parent | prev [-]

No, it's up to the people to regulate tech companies into submission.

Catloafdev 4 days ago | parent | prev | next [-]

They're creating optional opt-in protection layers for the services they operate.

I genuinely don't understand these generic complaint comments.

Are you complaining that they offer too much? Or do you believe nobody is offering similar services?

stuartjohnson12 4 days ago | parent [-]

The complaint is that the offer is a great deal with no downsides for consumers, and this is likely to result in Cloudflare having a lot of power (which they currently don't have) as a market maker. This position as market maker would grant them the power to extract economic rent from the web economy by charging both sides of the web provider and web consumer market to get access to the other.

Catloafdev 4 days ago | parent [-]

So the complaint is that one day they have such a strong monopoly that they can freely turn evil?

Just want to make sure I understand the real issue here, because that sounds like a lot of fearmongering to me.

stuartjohnson12 4 days ago | parent | next [-]

You don't need to speculate about whether a tech company with a monopoly over a major distribution channel will extract rent from it - it's incredibly profitable to do so.

pocksuppet 4 days ago | parent | prev [-]

Uhhh, they have it right now, and they are currently being evil by blocking a lot of people from accessing a lot of websites.

hoppp 4 days ago | parent | prev | next [-]

It's business as usual and it's our job to vote with our wallets.

jppope 4 days ago | parent | prev | next [-]

Agreed, but we should be honest, the internet today is far from healthy

cryo32 4 days ago | parent | prev | next [-]

Apart from handling of abuse reports. Yeah we're acting as CDN for this phishing site - we'll just inform the upstream about it and do nothing.

LoganDark 4 days ago | parent | prev | next [-]

It's really telling that they're starting to block anything automated ever that hasn't gone through their ID verification process. It's like literally every company that exists in the world is taking advantage of the dystopia at once

mark212 4 days ago | parent [-]

no, they're giving tools to their customers who can choose freely to block or not block bots. Without those tools, the people who run sites and offer content are just flying blind. I struggle to see how this is a bad thing in any way

LoganDark 4 days ago | parent [-]

My issue is with them defaulting every site to blocking any bot that hasn't gone through their verification process. It's not about the existence of the setting, but rather the behavior to enforce it by default.

ianm218 4 days ago | parent | prev | next [-]

For any one of their product there is a good opportunity to build an open source alternative or something like it! Can be hard to work around they have the benefit of being able to have negative unit economics on lots of infra products... But people succesfully built tons of alternatives to google analytics and similar.

esseph 4 days ago | parent | next [-]

What they are doing requires both physical and digital infrastructure spread throughout the globe. It's not a cheap task.

pocksuppet 3 days ago | parent [-]

Do you actually have customers spread throughout the globe, or just in North America and some in Europe?

ralegh 4 days ago | parent | prev [-]

Open source for bot protection specifically would be difficult. If I as a bot developer can see the tests you run I can just modify my bot to pass them (either trivially or by brute force).

whatjustin 4 days ago | parent | prev | next [-]

Unfortunately a common occurrence in the world. Not really a fan of cloudflare selling to both sides.

dzonga 4 days ago | parent | prev | next [-]

have you considered the alternative ?

where bots run rampant ?

trust me as an operator - I'm grateful Cloudflare exists.

Maxion 4 days ago | parent | prev | next [-]

To be frank, their products do work and are sorely needed.

baq 4 days ago | parent | prev [-]

...but very bullish NET. who wouldn't want to be the toll booth where you collect money both ways