> When I send a HTTP request to a webserver on the public internet it is up to them to decide if they want to respond to that request. And it is 100% up to me what I do with the data in that request on my machine in private.

Sorry, no. That's a software engineer's fantasy. And it's not even applicable to this case: it's not like Common Crawl is is only uses the data on their own machine in private, they're distributing it. Copyright is a real thing, that grants real rights, whether you like that or not. It doesn't disappear because computers and you don't like it.