| ▲ | jbellis 3 hours ago | |
It's not that hard to create your own uncontaminated eval for the problems you care about, and compare. Here's mine for coding in Rust, and I definitely see the same pattern of "most chinese models do much worse than on public evals that have been around long enough to train on" that AA v5 surfaces: https://slopcop.com/power-ranking | ||