Remix.run Logo
numbers 7 hours ago

I've stopped trusting these "trust me bro" benchmarks and just started going to LM Arena and looking for the actual benchmark comparisons.

https://arena.ai/leaderboard/code

stri8ted 7 hours ago | parent | next [-]

I doubt this is representative of real world usage. There is a difference between a few turns on a web chatbot, vs many-turn cli usage on a real project.

nba456_ 7 hours ago | parent | prev [-]

This is not any better of a benchmark