Then why the DDOS on random web sites?

I guess that depends on how the webspider is configured, I doubt the curation is done in real-time while scraping.