My grade school education in the US was similar to this (although you needed to qualify for "accelerated" math). There were maybe 90 people every year in this. Those who did well did not make it much past CML or AMC, maybe out of lack of interest. You had to be excellent to move on to Olympiad level maths.

Side note: I have many objections with the competition-ifaction of these things. Not everyone finds their best performance in a competition environment. In CS, maybe we would look at competitive programming and CTFs (arguably CP can be very mathematical). Nonetheless, we use competitions to measure performance, which in some way selects out people who are talented in a setting that isn't a competition, and glorifies people who do play the game.