That's a 2-bit quant of DS4 flash. You're probably better off running Qwen3.6-27B at Q8.

spmurrayzzz an hour ago | parent | next [-]

I think its good advice to test both on your own evals for sure, but the MoE parameters are already natively FP4 in ds4. Dropping to 2bpw isn't as big of a loss as it seems (and as corroborated by antirez's work).

Its also only 13B active, so your decode speed would be nearly 2x that of Qwen3.6-27B. So there are other latent benefits as well.

	▲	verdverm 11 minutes ago \| parent [-]
		z-lab has been dropping dflash addons for a lot of models https://huggingface.co/collections/z-lab/dflash I'm running the qwen3.6-27B + dflash on a spark and tgen is way up, but keep the number low, acceptance rate is terrible beyond half a dozen and it requires more memory

▲

Catloafdev an hour ago | parent | prev | next [-]

For most coding or agentic tasks, Qwen 3.6 27B likely outperforms, yes.

For 'general intelligence', DS4 Flash seems to be a noticeable step up still.

▲

sosodev an hour ago | parent | prev | next [-]

I suspect it would depend on the task. DS4-flash does, as previously mentioned, handle quantization very well. Even at 2-bit it's still very coherent.

▲

ckocagil 39 minutes ago | parent | prev [-]

Isn't Q8 way overkill these days? I see many graphs showing Q4 or Q5 having less than %1 deviation. Nvidia's NVFP4 Qwen quantization should be even better due to its better training methods.

	▲	mdgld 36 minutes ago \| parent [-]
		It depends on model size I think, but yeah, from my understanding at ~30B and below Q6 or even Q4 will get you 95%+ of the way there