Remix.run Logo
▲ the_duke 2 hours ago

To be fair, a lot of the popular benchmarks have become pretty meaningless.

Models often get great scores despite being a lot worse than the frontier in practice.

▲anon373839 22 minutes ago | parent [-]

This is true, but if you look at the change in weighting to specific benchmarks from one version of AA to the next, it looks really arbitrary.