Remix.run Logo
jrochkind1 4 days ago

What's now at the top has links to IETF drafts in the first paragraph. What am I missing?

A way to authenticate identity for crawlers so I can allow-list ones I want to get in, exempt them from turnstile/captcha, etc -- is something I need.

I'm not following what makes this controversial. Cryptographic verification of identity for web requests, sounds right.

binarymax 4 days ago | parent [-]

I think about failure modes. What happens if cloudflare decides you are a bot and you’re not. What recourse do you have? What are the formal mechanisms to ensure a person is not blocked from the majority of the web because cloudflare is a middleman and you are a false positive?

jrochkind1 4 days ago | parent | next [-]

I am not following what any of that has to do with the Web Bot Auth protocol?

it seems like complaints about Cloudflare's anti-DOS protection services and how they have a monopoly on such, I get that.

I'm not seeing the connection to a protocol for bots/crawlers voluntarily cryptographically signing their http requests, so sites (anyone implementing the protocol not just cloudflare) can use it to authenticate known actors?

I am interested in using it to exempt bots/crawlers I trust/support/have an agreement with from the anti-bot measures I, like many, am being forced to implement to keep our sites up under an enormously increased wave of what is apparently AI-training-motivated repeat crawling. Right now these measures are keeping out bots I don't want to keep out too. I would like to be able to securely identify them to let them in.

delroth 4 days ago | parent | prev | next [-]

Don't use a user agent that sends signed headers identifying you as a bot? How are any of the failure modes you mention not /improved/ by the spec proposal this comment section is about?

justincormack 4 days ago | parent | prev [-]

This is not a spec sbout false positives, ir is about self identification as a bot.