| ▲ | xscott 7 hours ago | |
It won't satisfy the people who just want to drop a model into their existing toolset and run, but I think there are a lot of ways to deal with this overthinking problem. For instance, it's a step backward, but I put {"reasoning_effort":"none"} and led it by the nose:
Maybe this can be fixed with Jinja templates or something, or maybe it's a hack to your harness, but it shows you can get the model to reason reasonably. | ||
| ▲ | vanviegen an hour ago | parent | next [-] | |
My impression is that when you allow the model to use internal thinking as opposed to asking it to output its thinking first, it's more likely to backtrack when I detects a flaw in its plan. Said otherwise: once producing user-facing output it seems to lock into an approach, for better or worse. | ||
| ▲ | theshrike79 an hour ago | parent | prev | next [-] | |
I feel that local models are better for "processes" where you need a degree of predictability. Like summarising the daily weather for the family chat bot or analysing email inbox priority. SOTA cloud models are more for open-ended tasks where you need "creativity". | ||
| ▲ | adam_arthur 6 hours ago | parent | prev | next [-] | |
Yes, if you set reasoning to none you can force the granularity of the thinking. It will actually adhere to your request for e.g. 3 sentences max. Thinking mode will override any instructions in the prompt (at least for other models in my experience). Of course this will probably hurt performance, but works great for easy tasks that you know are trivial. Tons of pipeline, image recognition etc use cases where this works well. I'd be curious to see Qwen 3.8 27B low thinking benchmarks though. | ||
| ▲ | hedgehog 6 hours ago | parent | prev | next [-] | |
To be fair a lot of models have quirks, I've never found a model swap that was transparent. | ||
| ▲ | regexorcist 3 hours ago | parent | prev [-] | |
I'm doing much the same, avoid the long thinking loops and instead have more iterations on the plan with reviews from different angles. | ||