Remix.run Logo
simonw 4 days ago

Yeah, wow that's a lot of information for a site that's supposedly blocked using robots.txt!

My best guess is that this is a Bing thing - ChatGPT uses Bing as their search partner (though they don't make that very obvious at all), and BingBot isnt't blocked by your site.

I think OpenAI need to be a whole lot more transparent about this. It's very misleading to block their crawlers and have it not make any difference at all to the search results returns within ChatGPT.