Remix.run Logo
▲ seizethecheese 5 hours ago

We allow early voting. This is against what most arenas do, but is on purpose. If the model is faster to do simple stuff, it should be chosen. In this sense, DS4.1-Flash is better on many tasks.

You can remove early voting from leaderboard and they are no longer the same.

▲nostrebored 38 minutes ago | parent [-]

But how do you verify a fair or representative distribution of complexity?