From what I've seen, when operating in an environment with bash tools, Claude and ChatGPT will react to a site blocking their normal, attributed fetch tool by calling curl or writing a python script with a different user agent to try and bypass the blocks, rather than respecting the site owners' wishes.

It'd be one thing if they required the user to consciously say "yes, I want you to bypass the block" (still unethical), but the fact that it's just default behavior really says something about the service design alignment.

Claude and ChatGPT are user agents. For obvious reasons, they serve the user, not the websites' owners. (Also, if you don't want people to be able to access some information, then don't put it online.)