Remix.run Logo
nsingh2 4 hours ago

Also with Qwen 3.8 being more token hungry than Luna, using around 2.3x tokens. Which hurts for local deployment.

sottol 4 hours ago | parent | next [-]

I'm torn on this - on the one hand performance matters, on the other so does capability.

I could run Qwen 3.6 27B on my laptop, but at 5 tok/s it was too slow even without overthinking - I never used it. OTOH, Qwen 3.6 35B A3B ran at 20 tok/s but it just could not get done what I asked of it. It sort of got close but you had to repeat and retry so much that it might have been faster to run 27B dense... maybe?

So that said, I might take a much better model that runs 2-3x slower (total time per task) but that's more capable over a faster, less capable one.

I'd also like to try a proper "plan-then-execute" type execution where thinking is entirely disabled (or low) during the execution stage but enabled/max during the planning stage.

I will definitely give 3.8 27B a better shot than 3.6 though.

skohan 3 hours ago | parent | prev [-]

Depends on your use-case. Over the past couple days, I've found 192k context more than enough for coding. There's more thinking for sure compared to comparably sized models (running on xhigh), but I've found the results are so much better that the entire session consumes less tokens on average since weaker models need more review passes.