| ▲ | nharziro 7 hours ago |
| I do agree that Qwen 3.8 27B is excellent but slow and very token inefficient. My benchmark places it near opus 4.6 and codex 5.3 performance. 3.6 27B couldn't even complete the benchmark. Please see below for details: https://gist.github.com/nharziro/aed0c364ce2f295a493494c6f1b... |
|
| ▲ | matheusmoreira 5 hours ago | parent [-] |
| Opus 4.6 performance with a local model that can be hosted on consumer hardware is an incredible result!! |
| |
| ▲ | nharziro 4 hours ago | parent [-] | | I was genuinely surprised because it's quite a leap from where 3.6 was an as far as I understand this isn't a new model, it's the same model that's been post trained, so I don't quite understand what they did to improve it so substantially. The previous model couldnt get through the benchmark at all. Though it remains terribly inefficient and slow. The hardware will have to get substantially faster for these kinds of models to be daily drivers. I think I forgot to mention in the bench that I ran it on an m5 max mac book |
|