I am aware of all of the things you mention (I've built crawlers before).

My point was only that there are plenty of crawlers that don't operate in the way the parent post described. If you want to call them buggy that's fine.