What’s that about the template breaking the kv cache?

this is my understanding, the default template keeps the thinking part but only for the last message, so the harness has to play along with the template and strip and add to keep the conversation matching what's there on the llama.cpp cache, but if the harness sends the thinking in every turn, then you break what llama.cpp expects, the conversation doesn't match anymore what you have on cache and it reprocesses again the whole conversation

▲

Gracana 6 hours ago | parent [-]

Oh, interesting. I want to say there’s a “preserve thinking” option in mainline llama.cpp, but I’m seeing mixed results in my searches and my in-flight internet is not good enough to dig into it properly. Good to hear you have a solution though, because that is likely something I’ll run into as well.

	▲	hedgehog 6 hours ago \| parent [-]
		Look for improved templates by "froggeric" on Hugging Face. I use Qwen 3.6 a fair amount using his template and it fixes some issues I saw with the upstream versions.