| ▲ | rvz 2 hours ago | |||||||
I am going to assume that this also destroys millions of AI agents and bot scrapers this time which is why some “AI Engineers” were complaining about this recently. Well, this is how Google will kill all the scrapers on its search data. | ||||||||
| ▲ | jeroenhd 2 hours ago | parent | next [-] | |||||||
Not entirely, Google's own page says: > Fraud Defense leverages a sophisticated and adaptable risk analysis engine to shield against automated software. It is specifically designed to orchestrate trust for the agentic web, neutralizing malicious scrapers while welcoming legitimate AI agents. I'm sure it'll block a whole bunch of awful scrapers but if Google doesn't hate a bot, it'll be able to pass. | ||||||||
| ||||||||
| ▲ | hsbauauvhabzb an hour ago | parent | prev [-] | |||||||
Does mass scraping need google for content discovery? Surely most sites contain a site map or index that would effectively self enumerate once you know the domain, which is more often than not publicly disclosed? | ||||||||