What configuration are you using? On both vllm and llama-cpp, I get significantly higher speeds from gemma4 than qwen3.6 (with their respective speculative decoding methods).

Output TPS in vllm for instance:

- Gemma4 26B-A4B: 200-300TPS

- Qwen3.6 35B-A3B: 120-180TPS

- Gemma4 31B: 80-120TPS

- Qwen3.6 27B: 60-80TPS

This is for a first request on a dual 5090 setup, with their respective speculative decoding methods.

Have you tried running it on a single 5090? Dual 5090 require https://github.com/aikitoria/open-gpu-kernel-modules for higher perf. Are you using TP?

Single 3090 under llama.cpp:

  | model               |    size |   test |  t/s |
  | ------------------- | ------- | ------ | ---- |
  | gemma4 31B Q4_0     | 16.1 GB | pp2048 | 1248 |
  | gemma4 31B Q4_0     | 16.1 GB |  tg512 |   40 |
  | qwen35 27B Q4_K     | 15.9 GB | pp2048 | 1248 |
  | qwen35 27B Q4_K     | 15.9 GB |  tg512 |   39 |
  | gemma4 26B.A4B Q4_0 | 13.3 GB | pp2048 | 4304 |
  | gemma4 26B.A4B Q4_0 | 13.3 GB |  tg512 |  160 |
  | qwen35 35B.A3B Q3_K | 15.7 GB | pp2048 | 3329 |
  | qwen35 35B.A3B Q3_K | 15.7 GB |  tg512 |  144 |

You're benchmarking drafter acceptance rate, then. Which is real life values, yes, but attributing worse drafter performance to the other 95% of the model being inherently slower.