One interesting reason this happens, at least in the music field, is the adult disadvantages that often go along with various forms of savantism. I have spoken with a number of fellow music academics about this, and it's not uncommon that the things that make one a young prodigy are the same things that give one real obstacles to making it in the regular world, and this can impose a ceiling on where they get to. For example, many music prodigies have never "really had to work" and once they get to having to shoulder the boring reponsibilities that go with building a career, they instead alienate people, or just can't do things that are hard for them because it's always been easy. Maybe they can't play off charts unless they've heard it, or aren't used to following instructions/cues/being the lead, etc. And unless they are truly, truly rare air, real career gigs have boring work elements too.

Savantism can be pretty damned weird. I've known a few, including a couple who will never have an adult career beyond local gigs because of their mental disabilities in other, non-music areas. The Oliver Sacks book "Musicophelia" has fascinating case stories about it.