Remix.run Logo
▲ ariwilson 5 hours ago

Sorry this was confusing -

There are 30 tasks in my eval total:

- 20 of them ran on Nov 1, twice each (so 40)

- I created 10 more tasks and ran them on Nov 5 (also ran them twice each)

- Today I ran all 30 tasks, once each, on Opus 5.5 to see if the performance decreased at all

- I only put the comparison against the original 20 in my post, since I saw in another post that people are claiming Opus was nerfed on November 3

This will all get less confusing when I make my benchmark public (hopefully this week).