Do you think that LLM leaderboards don’t? Do you think a Llama 3 is going to beat an Opus 4.7 on any leaderboard?

The real issue is that standardized tests disenfranchise lower SES students less than any other metric.

Everyone who takes the SAT has to sit in the same room for the same amount of time answering the same questions. You can’t just pay someone else to take it for you (like essays) or select which difficulty level you take (like going to a prep school with grade inflation), or luck out in who your parents know (like recommendation letters).

Some may have better opportunities to learn the material, but, at the end of the day, you have to actually learn the material. There’s no getting around that.

As your own GRE anecdote shows: A little studying with some inexpensive books makes all the difference. Unless things have radically changed, a couple SAT or GRE test prep books are significantly less expensive than just one college textbook.

Bluntly, the reason SATs are better correlated to college performance than other measures are because of the reasons I mentioned. They strip away most of the privilege of coming from a high-SES family.