I'd guess it's due to some kind of imprinting during childhood, similar to taste. The widespread prevalence of irrational phobias and methods for curing them certainly suggest to my untrained eye a learned behavior rather than innate.