Very much disagree, to me it clearly shows the gap between Open Source and the frontier. You can not do real code work with open source models right now.

Especially since this benchmark is still mostly DeepSWE, and the only reason the open source models are doing decent is because much of the information hasn't changed. The open source models have been benchmaxxed to death.

> Very much disagree, to me it clearly shows the gap between Open Source and the frontier.

Yes, this is true.

> You can not do real code work with open source models right now.

This is false, or you're using "real" as a loophole. You can absolutely do real coding work with open source models, right now, right this second.

I'm doing so today using Qwen 3.8 Flash Next running locally, and GLM 5.3 Flash hosted.

We could do real work with Opus 4.5, 4.6. These models (for coding) are more than good enough to do work that Opus 4.6 could do, and in some ways are superior.

Even if frontier models are much stronger than open weights, strong disagree on this. You very much can get real work done using open weights models.

People who use mythos for everything also say you cant get real work done with opus.

You just adapted to niceties of your chosen model.

> You can not do real code work with open source models right now.

I only use open weights and get plenty of real work done. Turns out most real tasks are not that hard, rather they are tedious.

[dead]