This is true, but I think it understates the problem. I did a task I've done with a bunch of small models lately (https://github.com/swelljoe/flar/pull/17), and it did an excellent job, the best of any self-hostable model. But, it took eleven (11!) hours on my dual GPU setup. It really chewed on it, and spent a lot of time checking and re-checking. It is by far the slowest model I've used for the task. GPT 5.5 did a similar task in about 20 minutes. Most big models took about an hour or so, and most small models needed a couple of hours (but did a worse job).

▲

simonw 7 hours ago | parent | next [-]

Was that with the default xhigh reasoning setting? I suggest trying again with reasoning set to low or turned off entirely.

▲

dofm 6 hours ago | parent | next [-]

You now have me testing it with reasoning turned off, which I have never bothered much with on any other local models because it's rarely worth it.

The result appears to be almost as good as Qwen 3.6 35B A3B on medium thinking mode.

It second-guesses a little, it gives broader/more speculative answers, of course, and it missed the nuance of one of my prompts, but this gives me a lot more confidence that the Low reasoning effort is going to be as good as they say, and perhaps in some cases non-thinking looks like it would be enough.

Really useful, thanks.

▲

anon373839 5 hours ago | parent [-]

> Qwen 3.6 35B A3B on medium thinking mode

Qwen 3.6 doesn’t have configurable reasoning effort, does it?

	▲	dofm 4 hours ago \| parent [-]
		Hm — brain jumped tracks a bit there at nearly 4am. I'm talking about budget — I mean limiting it to 2048 tokens. … for one or other of the models I tested at the same time, in llama-server, there is a dropdown that offered options (unlimited, max, medium which was 2048) (I've tested so many of these things now that they are beginning to blur.) I thought that was llama-server with Qwen 35B, just checked and it's not. Nor is it Gemma 4 26B. Perhaps it was Ternary Bonsai which I tested again and deleted earlier. Anyway I took to clipping Qwen 3.6 35B at 2048 tokens reasoning in LM Studio and elsewhere, and it did OK at that (because it often loops like mad on an ambiguous prompt if not curtailed). FWIW I just rechecked outputs and I am a bit over-optimistic when I say 3.8 27B 's non-thinking output is that good. I spotted a couple of subtle errors in my tests that Low thinking didn't fail on. It is good, but it is not quite Qwen 3.6 35B thinking level.

▲

SwellJoe 7 hours ago | parent | prev [-]

Yes, default everything, no tuning, 8_K_XL Unsloth quantization on dual Radeon V620 GPUs (which aren't blazing, but faster than the Strix Halo).

▲

syntaxing 6 hours ago | parent [-]

What speed do you get on this setup? Im tempted to use the same GPU.

	▲	SwellJoe 5 hours ago \| parent [-]
		It varies widely based on a bunch of factors. With this specific model at 8-bit quantization and MTP, it starts out at about 25 t/s for basic chat, but for agentic tasks with long context it slows way down to something like 12-15. I don't see a big difference in token rates based on any config changes I have tried, or going to the smaller 6-bit quantization, so far, though I haven't spent a ton of time on experimenting. If you already have one or more of them, then, yeah, you can use them for this model or any other at around this size or smaller, but I wouldn't recommend you buy them (or anything else, right now, everything is just too overpriced). You can run better models for less money at higher speeds. I bought mine before they got more expensive, but I wish I'd just bit the bullet and bought newer/faster cards before they got more overpriced. Or, the actual smart money, even back then was to just use cloud models and forget about self-hosting.

▲

fermuch 7 hours ago | parent | prev [-]

xhigh tells it to overthink and re check everything. Low tells it to only do the minimum thinking necessary. I would suggest to give qwen medium which doesn't inject any thinking directives into it and also to give as much context as you can, ideally around 500k tokens or even 1M if you can. Big complex tasks like these make the model hit the compaction trigger a lot and they end up re thinking the same thing several times in my experience.

▲

kennywinker 5 hours ago | parent [-]

Doesn’t it max out its context at like 256k?

	▲	SwellJoe 4 hours ago \| parent [-]
		It supports YaRN. From the model card: "Context Length: 262,144 natively and extensible up to 1,010,000 tokens." But, ~256k is all I can fit on my dual 32GB setup, and it gets slow as heck at 256k, anyway. I think just convincing it to chew less and do more is probably the right solution for Qwen 3.8 27B, though I guess we need some benchmarks to get the feel for how much intelligence you lose by choosing a lower reasoning level.