Remix.run Logo
▲ seizethecheese 20 minutes ago

I was skeptical Sol 6.1 actually over-engineers compared to Opus 5.5, as I just switched from one to the other and haven't noticed a difference, but it looks like you're probably right!

We're building a battle system for OpenCode (https://agentduel.com/leaderboard) and have data on how large the diff was compared to opponents for both:

Opus 5.5: 0.42x (based on 299 battles) Sol 6.1: 1.42x (based on 265 battles)

Data on this is still pretty small. P=0.14. It's also affected by large differences, this is average not median.

▲seizethecheese 4 minutes ago | parent [-]

Oh man, Sol goes nuts on tests. I just checked the test wriging data. Sol 6.1 writes 3.14x more tests than it's opponents.