Remix.run Logo
hellajack3d an hour ago

I forked llama.cpp and added some crude mechanism to keep exactly this behavior under control - essentially guiding the reasoning process by injecting text strategically at specific thresholds. This was mainly put together to rein in Qwen3.6-27B, but I'd imagine 3.8 would react similarly.

Fork can be found here - https://github.com/laurencehardman/llama-mindcontrol/tree/ma...

Of course hacks like this are not perfect and may degrade performance slightly due to injected text pushing the model slightly out-of-distribution, so the string constants need to be chosen carefully - Qwen3.5's technical whitepaper does provide some guidance in this regard. The mechanism is absolutely more of a hack than a feature, and i'd imagine will be made redundant once llama.cpp supports more appropriate reasoning controls - but for now, i've found it pretty useful.

DarmokJalad1701 an hour ago | parent [-]

Is that similar to what ggerganov is talking about here?

https://x.com/ggerganov/status/2089214161884414147

hellajack3d an hour ago | parent [-]

Yes - it would seem so :)

I did make a PR to the official llama-cpp repo some time back (about a month or so), but abandoned it as there seemed to be too much community concern that the mechanism would degrade model performance... Perhaps i'll polish it up and put some effort into benchmarking and revive the project in the near future.