I was trying to use a grain/chaff analogy to respond to your post, but I think there were just less crops in the old days. For the sites (crops) that were there, you had a lot more healthy ones. As spam and low-quality sites proliferated, the signal->noise ratio of sites got completely out-of-balance.