| ▲ | seizethecheese 20 minutes ago | |
I was skeptical Sol 6.1 actually over-engineers compared to Opus 5.5, as I just switched from one to the other and haven't noticed a difference, but it looks like you're probably right! We're building a battle system for OpenCode (https://agentduel.com/leaderboard) and have data on how large the diff was compared to opponents for both: Opus 5.5: 0.42x (based on 299 battles) Sol 6.1: 1.42x (based on 265 battles) Data on this is still pretty small. P=0.14. It's also affected by large differences, this is average not median. | ||
| ▲ | seizethecheese 4 minutes ago | parent [-] | |
Oh man, Sol goes nuts on tests. I just checked the test wriging data. Sol 6.1 writes 3.14x more tests than it's opponents. | ||