MIT's PRIMES program does exactly this - they give advanced high school students a mentor who picks out a problem, gets them up to speed on what is known, and then they work on the problem for a year and publish their results. It tends to work best with problems which have a computational aspect, so that the students can get some calculations done on the computer to get the ball rolling.