With bots that break through cloudflare captchas. As a human, I immediately close the browser tab when I see one. If you make your site inaccessible for humans, only the bots remain.

I was more so asking how does it know where to look to find information, if things are no longer being indexed.