With stuff like https://www.cloudflare.com/en-in/application-services/produc... and https://blog.cloudflare.com/ai-labyrinth/ big money going on both sides last thing you want is to shadow detected as a bot. Its all ok if you are scraping to top rated SEO slop which is usually static sites but for anything beyond it wont work well eventually. Quite a few issues on browerbase, crawl4ai and similar repos around being detected as a bot.