> So call them out for the bad stuff!

I am. I am also saying -because the companies involved have demonstrated that they're either frequently willing to do things that are scummy as shit or "just" have executives that make a habit of lying to the press in order to keep the hype train rolling- that's it's very, very likely that they're quietly engaging in antisocial behavior in order to make development of their projects some combination of easier, quicker, or cheaper.

> Don't distract from the genuine problems by making up stuff...

Right back at you. You said:

  > There are definitively scrapers that ignore your robots.txt file
  Of course. But those aren't the ones that explicitly say "here is how to block us in robots.txt"

The unfortunate thing for us is that neither of us can do anything other than speculate... unless an insider turns informant.

[0] ...and given how the expected penalties for engaging in most of the antisocial behavior that's relevant to the AI research project is somewhere between "absolutely nothing at all" and "maybe six to twelve months of expected revenue"...