The intent is to not make companies shoulder the cost of other organizations scraping their content. When it is regular users browsing the cost incurred is trivial. When bots are scraping the entirety of a site, repeatedly, it adds up quickly.