Remix.run Logo
jasonjmcghee 5 days ago

Very interested to see the updated results. This could really shake up the leaderboard.

macawfish 5 days ago | parent [-]

I hope it does. These coding benchmarks have often seemed frustratingly out of touch with my experience.

typpilol 5 days ago | parent | next [-]

The agentic ones seem better. Typescript is like at 25% last I saw on the models. Python was higher.

That seems more accurate than the huge scores the other ones get

3abiton 5 days ago | parent | prev [-]

Because I would argue there is no benchmark to rule them all. It highly depends on individual use cases.