▲ | simonw 4 days ago | |
Yeah, wow that's a lot of information for a site that's supposedly blocked using robots.txt! My best guess is that this is a Bing thing - ChatGPT uses Bing as their search partner (though they don't make that very obvious at all), and BingBot isnt't blocked by your site. I think OpenAI need to be a whole lot more transparent about this. It's very misleading to block their crawlers and have it not make any difference at all to the search results returns within ChatGPT. |