| ▲ | cg5280 2 days ago | ||||||||||||||||
I like the inclusion of the graph at the end to compare progress. It would be cool to compare this directly to competing models (Claude, GPT, etc). | |||||||||||||||||
| ▲ | kqr 2 days ago | parent [-] | ||||||||||||||||
It would unfortunately also need several runs of each to be reliable. There's nothing in TFA to indicate the results shown aren't to a large degree affected by random chance! (I do think from personal benchmarks that Gemini 3 is better for the reasons stated by the author, but a single run from each is not strong evidence.) | |||||||||||||||||
| |||||||||||||||||