> Qwen models are slower in tokens/s, compared to similarly sized gemma4 and others

No? Gemma 31B and Qwen 27B are about the same speed. Gemma 26B-A4B and Qwen 35B-A3B are about the same speed.

▲ trouve_search 3 hours ago | parent | next [-]

What configuration are you using? On both vllm and llama-cpp, I get significantly higher speeds from gemma4 than qwen3.6 (with their respective speculative decoding methods).

Output TPS in vllm for instance:

- Gemma4 26B-A4B: 200-300TPS

- Qwen3.6 35B-A3B: 120-180TPS

- Gemma4 31B: 80-120TPS

- Qwen3.6 27B: 60-80TPS

This is for a first request on a dual 5090 setup, with their respective speculative decoding methods.

	▲	xfalcox 6 minutes ago \| parent \| next [-]
		Have you tried running it on a single 5090? Dual 5090 require https://github.com/aikitoria/open-gpu-kernel-modules for higher perf. Are you using TP?
	▲	petu 2 hours ago \| parent \| prev [-]
		Single 3090 under llama.cpp: `\| model \| size \| test \| t/s \| \| ------------------- \| ------- \| ------ \| ---- \| \| gemma4 31B Q4_0 \| 16.1 GB \| pp2048 \| 1248 \| \| gemma4 31B Q4_0 \| 16.1 GB \| tg512 \| 40 \| \| qwen35 27B Q4_K \| 15.9 GB \| pp2048 \| 1248 \| \| qwen35 27B Q4_K \| 15.9 GB \| tg512 \| 39 \| \| gemma4 26B.A4B Q4_0 \| 13.3 GB \| pp2048 \| 4304 \| \| gemma4 26B.A4B Q4_0 \| 13.3 GB \| tg512 \| 160 \| \| qwen35 35B.A3B Q3_K \| 15.7 GB \| pp2048 \| 3329 \| \| qwen35 35B.A3B Q3_K \| 15.7 GB \| tg512 \| 144 \|` > with their respective speculative decoding methods You're benchmarking drafter acceptance rate, then. Which is real life values, yes, but attributing worse drafter performance to the other 95% of the model being inherently slower.

▲ stymaar 3 hours ago | parent | prev [-]

There's no Qwen3.8-35B-A3B though.

	▲	hadlock 2 hours ago \| parent [-]
		I benched Qwen 3.6 35B-A3B against Qwen 3.8 27B with the same parameters, thinking set to low. Despite 35B having 9x fewer active parameters, it benched only 2.34x slower. The 35B got only 50% more agentic tasks done per hour.