Isn't Q8 way overkill these days? I see many graphs showing Q4 or Q5 having less than %1 deviation. Nvidia's NVFP4 Qwen quantization should be even better due to its better training methods.

It depends on model size I think, but yeah, from my understanding at ~30B and below Q6 or even Q4 will get you 95%+ of the way there