But a massive number of the bay area "weirdos" seem to come from elite schools; or is my frame of reference not representative?

Maybe is just the concentration of technical talent (usually introverts, home buddies), whom put less emphasis on social skills.