Qwen models are slower in tokens/s, compared to similarly sized gemma4 and others, and they use more tokens per task, in part thanks to that xhigh default.

On the other hand, there are some of us who are stuck with hardware that has plenty of compute, but limited (V)RAM. The new 27B is just perfect for that.

▲ petu 4 hours ago | parent [-]

> Qwen models are slower in tokens/s, compared to similarly sized gemma4 and others

No? Gemma 31B and Qwen 27B are about the same speed. Gemma 26B-A4B and Qwen 35B-A3B are about the same speed.

▲ trouve_search 3 hours ago | parent | next [-]

What configuration are you using? On both vllm and llama-cpp, I get significantly higher speeds from gemma4 than qwen3.6 (with their respective speculative decoding methods).

Output TPS in vllm for instance:

- Gemma4 26B-A4B: 200-300TPS

- Qwen3.6 35B-A3B: 120-180TPS

- Gemma4 31B: 80-120TPS

- Qwen3.6 27B: 60-80TPS

This is for a first request on a dual 5090 setup, with their respective speculative decoding methods.

	▲	xfalcox 7 minutes ago \| parent \| next [-]
		Have you tried running it on a single 5090? Dual 5090 require https://github.com/aikitoria/open-gpu-kernel-modules for higher perf. Are you using TP?
	▲	petu 2 hours ago \| parent \| prev [-]
		Single 3090 under llama.cpp: `\| model \| size \| test \| t/s \| \| ------------------- \| ------- \| ------ \| ---- \| \| gemma4 31B Q4_0 \| 16.1 GB \| pp2048 \| 1248 \| \| gemma4 31B Q4_0 \| 16.1 GB \| tg512 \| 40 \| \| qwen35 27B Q4_K \| 15.9 GB \| pp2048 \| 1248 \| \| qwen35 27B Q4_K \| 15.9 GB \| tg512 \| 39 \| \| gemma4 26B.A4B Q4_0 \| 13.3 GB \| pp2048 \| 4304 \| \| gemma4 26B.A4B Q4_0 \| 13.3 GB \| tg512 \| 160 \| \| qwen35 35B.A3B Q3_K \| 15.7 GB \| pp2048 \| 3329 \| \| qwen35 35B.A3B Q3_K \| 15.7 GB \| tg512 \| 144 \|` > with their respective speculative decoding methods You're benchmarking drafter acceptance rate, then. Which is real life values, yes, but attributing worse drafter performance to the other 95% of the model being inherently slower.

▲ stymaar 4 hours ago | parent | prev [-]

There's no Qwen3.8-35B-A3B though.

	▲	hadlock 2 hours ago \| parent [-]
		I benched Qwen 3.6 35B-A3B against Qwen 3.8 27B with the same parameters, thinking set to low. Despite 35B having 9x fewer active parameters, it benched only 2.34x slower. The 35B got only 50% more agentic tasks done per hour.