| ▲ | tripzilch 12 hours ago | |||||||
> running Qwen 3.6 35b a3b sipping 10w when generating tokens is good enough for a lot of my guide-coding work can you tell more about how you're using it? like, what harness? or also in the IDE? I found Qwen3.6 35B/A3B to make slightly too many mistakes (already in its harness' tool use, hence my question), maybe it gets the job done, but it will also sometimes generate a bit of a mess (e.g. editing/creating files in the wrong folders) and fixing/solving its own mistakes takes time (or tokens) .. | ||||||||
| ▲ | Phemist 5 hours ago | parent | next [-] | |||||||
I feel like ornith1.5 35B/A3B is an overall stronger model on the same architecture, so a drop-in replacement untill qwen3.8/qwen4 is released. Using the 8bit quant on my M4 max gets around 80tok/sec output/decode on an empty context, dropping down to 35ish on nearly full one. | ||||||||
| ▲ | jorgeleo 11 hours ago | parent | prev [-] | |||||||
Same thing for me. on an M5 max, Qwen 3.6 35b gives me between 150 and 200 tps using splash as inference engine. More than enough for guided code sessions, at 100% privacy. And i can use obliverated models if i am trying to harden my own app, something i cannot do with cloud providers. | ||||||||
| ||||||||