Did you have rate limits built in? Ultimately scraping tools will need to mimic humans. Ironic.

I wonder if bots/ai will need to build their own specialized internet for faster sharing of data, with human centered interfaces to human spaces.

IPFS and IPNS already exist.