I agree and I think that everyone agreeing or disagreeing with you (and sysadmins everywhere) would be perfectly fine with these AI crawlers (well, mostly...) if these corporations wrote them properly, followed best practices and standards, and didn't effectively DDoS servers or pretend to be what they aren't. Because that is, ultimately, what these AI companies are: very effective, for-sale, legal DDoSers. But they are not written properly, do not follow best practices and standards, and DDoS everything you aim them at, and even go as far as pretending that they're things they aren't, hide behind residential IP addresses (which I'm pretty sure could potentially be illegal because, you know, that risks getting people who have no idea what AI even is in trouble), etc. I don't think AI will replace search now just because so much of the world is blocked from them now, and that is only to increase I'm sure. And, honestly, I doubt there is anything these AI companies could do to make sysadmins actually trust them again anymore.