Well it's bad compared to the original model, but is it bad compared to what else you could run in 64 GB?

probably; 32-64 is the range where local models get extraordinarily useful, and in my experience any sub-4 quants get ridiculously incoherent, which is why those models tend to try to 'wow' people with tok/s rather than benchmarks.

haven't tried it yet though.

	▲	cyanydeez 3 hours ago \| parent [-]
		Qwen3.8-Flash-Next fits in 80GB on strix halo. That's when I started real "vibe" coding with them. Speed is still an issue if I want to move fast and break things, but I've built a little universe for development (I assume many people have also) and it's cruising. But yeah, 32-64 is the gateway I think. If Qwen4 can make a leap in the flast-next MoE and hit 64 GB on VRAM w/context, that's definitely the cloud killer to me.

▲

Tepix 5 hours ago | parent | prev | next [-]

I’m wondering myself: Do bigger models suffer less from quantization? Are there papers?

▲

ricardobeat 4 hours ago | parent [-]

They seem to degrade a lot less.

You don't need papers, good quants are usually published with benchmark data, for example: https://huggingface.co/ISTA-DASLab/DeepSeek-V4.1-Flash-1017-... which maintains parity with FP8 even at 2 bit.

	▲	alfiedotwtf 3 hours ago \| parent [-]
		Always check the label and if pain persists, see your doctor! The ISTA accuracy looks good on that page, but it's only a chosen handful. Instead, look at most of the graphs on Unsloth's Qwen 3.8 Flash Next - https://unsloth.ai/docs/models/qwen3.8-next As always, check the quant vs KLD graphs (half way down that page).. I found that not all quants are the same - Unsloth's GLM Q4 -> Q3 degrades a lot faster than their Minimax M3.1 Q4 -> Q3 ggufs.

▲

pitchlatte 4 hours ago | parent | prev [-]

you can get a lot done with models well under that - the google and qwen 20-30B class models are fantastic