| ▲ | reincoder 4 hours ago | |
I used the same methodology to observe AI crawlers. This is not an investigative blog but is rather designed to address our (IPinfo) customers who are asking us to identify IP addresses as "AI Agents" or, more accurately, "AI Crawlers". https://community.ipinfo.io/t/can-we-detect-ai-agents-we-can... Most AI crawlers self-identify with a UA. However, Grok uses resproxies and sends a high volume of simultaneous requests. Even though we can detect resproxies, it is not possible to map these resproxy IPs to grok. I still could not figure out why I saw legitimate Googlebot IPs when I requested Perplexity to review the website. I verified those Googlebot IPs using both using UA and the listed IP address ranges published by Google. | ||