| ▲ | kuylar 19 hours ago | |
> As for AI scrapers impersonating googlebot I feel like yes they'd definitely start doing that, unless the risk of getting sued by google is too high? Google releases the Googlebot IP ranges[0], so you can makes sure that it's the real Googlebot and not just someone else pretending to be one. [0] https://developers.google.com/crawling/docs/crawlers-fetcher... | ||
| ▲ | n1xis10t 19 hours ago | parent [-] | |
Oh good idea! | ||