| ▲ | hellajack3d an hour ago | |||||||
I forked llama.cpp and added some crude mechanism to keep exactly this behavior under control - essentially guiding the reasoning process by injecting text strategically at specific thresholds. This was mainly put together to rein in Qwen3.6-27B, but I'd imagine 3.8 would react similarly. Fork can be found here - https://github.com/laurencehardman/llama-mindcontrol/tree/ma... Of course hacks like this are not perfect and may degrade performance slightly due to injected text pushing the model slightly out-of-distribution, so the string constants need to be chosen carefully - Qwen3.5's technical whitepaper does provide some guidance in this regard. The mechanism is absolutely more of a hack than a feature, and i'd imagine will be made redundant once llama.cpp supports more appropriate reasoning controls - but for now, i've found it pretty useful. | ||||||||
| ▲ | DarmokJalad1701 an hour ago | parent [-] | |||||||
Is that similar to what ggerganov is talking about here? | ||||||||
| ||||||||