I personally think its easier to detect llm controlled browser sessions, the people deploying them are far more naive and inexperienced than traditional scrapers/crawlers.

insert You wouldn't bring a 40 Petabyte Zip Bomb to School, would you? meme

Part of the problem is also that Google wants to permit crawlers to do some things but jot others.

Their announcement is full of buzzwords about "agentic" things. Detecting LLMs is one thing, but imagine the power of being able to pick which LLM browsers are permitted and which aren't!

I think Google is being too early to the party with this. Cloudflare still has CAPTCHAs to throw at the wall. There are ways other than attestation to verify that someone is a real human, but they're getting more and more annoying to real users and harder and harder to implement on a small website.

Despite the massive implications, this is a simple system that just works for the 99% of people who use Chrome or Safari or at least have access to an Android phone or iPhone somewhere. It's quick, doesn't require installing apps or creating accounts, and it just works from both the website perspective and the user perspective.

Of course when you start thinking about people with disabilities things become problematic, but when have tech companies ever really cared about that sort of thing? Inclusiveness was fun and all for a while, but the clowns the American people elected banned that sort of thing for any company considering government contracts, and big tech licked that boot like it was made of honey.

The world becomes a lot easier if you just decide to ignore all edge cases and assume customers who disagree with you didn't matter anyway. And infuriating as it may be, for companies like Google, that business model works.