> If it takes them 100 times the average crawl time to crawl my site, that is an opportunity cost to them.

If it takes 100 times the average crawl time per page on your site, which is one of many tens (hundreds?) of thousand sites, many of which may be bigger, unless they are doing one site at a time, so your site causes a full queue stall, such efforts likely amount to no more than statistical noise.

Again, that delay is mostly about me, and my employer, not the rest of the world.

However if you are running a SaaS or hosting service with thousands of domain names routing to your servers, then this dynamic becomes a little more important, because now the spider can be hitting you for fifty different domain names at the same time.