With all due respect, there's zero chance that humans with relevant knowledge scored these themselves. Reading through the winners, every single one is classic vibe-research, with the usual pure-LLM-research patterns:

- Grand claims backed by no evidence

- Core designs that make zero sense

- Pointless graphs that show nothing of interest

- Yet endless robustness checks on minor methodological assumptions (especially confidence intervals and t-tests)

For example, on the winning entry, not only is the graph completely wrong (as mentioned by the OP), but the interpretation would be nonsensical even if it was (implying bigger models "get more RL"?). And their own results even show the core dataset is worthless, because all their metrics are near-perfectly correlated. There's no way a serious human reader trying to evaluate "is this benchmark useful" would ever miss this.

I don't mean to pick on them - all the winning entries seem like there was no human effort put into them. And again, there's no way a human who actually attempted to read and understand these would ever think these are good by even the most minimal of standards.

Kaggle is legitimately a really awesome website, as someone who's competed before and won a few contests pre-LLMs. Stuff like this winning devalues the entire product and makes it look like a joke. If almost all entries look like this now, it'd be better to allow for the possibility of no winner.