Except robots.txt was the actual real solution to search indexing...