new | show | ask | jobs Github

goldenarm 6 hours ago

Geometric mean of all these benchmarks :

* GPT-5.6 Sol: 65.5

* Fable 5 (w/ fallback): 64.5

* Opus 5: 64.0

* DS-V4-Pro 0813: 62.5

* Kimi-K3: 62.3

* DS-V4-Flash 0731: 55.8

* GLM-5.2: 47.3

▲

svachalek 5 hours ago | parent [-]

Maybe it's me but I don't see how DS Flash is better than GLM at all, much less by a huge gap. I'd probably protest less against Fable and Opus being put at the same level than many would, but there's no denying the two models are a very different experience from each other. I guess where I'm going is no one should pick a model by the benchmarks.

▲

spijdar 4 hours ago | parent | next [-]

I'm not the most LLM-savvy person around, and I'm not gonna say I've put a ton of effort into practically compared these open models. But, a month or two ago I did do some "practical evaluates" testing GLM 5.2 versus DSv4 (flash/pro) with OpenCode's subscription with some late 80s Unix clone-type work, and this jives with my experience.

GLM ended up being far slower, and far more expensive, for approximately the same results. There was never a problem that GLM could solve that DS couldn't solve, faster, and significantly cheaper.

I strongly agree that you shouldn't pick a model based on benchmarks. But for me, I found GLM really underwhelming given its cost and speed.

DSv4 isn't as good as GPT or Claude or what have you, but it's fast, and pretty darned effective. I can run a 3-bit quant of DSv4 locally on my system with ~15 tokens per second, and for a local model it might be the most overall effective at coding. For what it is, it's extremely impressive.

	▲	ApolloFortyNine 3 hours ago \| parent [-]
		My experience is the same. Imo it has a lot to do with you/the harness tries to get it to test itself. Deepseek v4 flash seems more than capable of understanding when something has failed, and making changes until it works. I've definitely seen it make mistakes I would expect something like Opus to find, but it works through them on it's own (and for literal pennies). At the end of the day, I think that's one of the most important features of a model.

▲

segmondy 3 hours ago | parent | prev | next [-]

It isn't. I run both at home. GLM5.2 Q4 crushes DSv4Flash0731 Q8. I reach for DS for speed and for medium effort level work. If I care about quality I'll reach for GLM5.2 Looking at this release, I'm comparing it to GLM5.2 and it seems to beat GLM5.2, only time/experience will show. If true, then I'm happy. It's much easier to run than Qwen3.8/KimiK3

▲

spiffytech 4 hours ago | parent | prev | next [-]

In my little social circle DS4F generally substitutes for GLM 5.2 except it's the next best thing to free.

▲

platinumrad 5 hours ago | parent | prev [-]

I think instruction following carries outsized weight in these evaluations.