As usual, the Jinja templates are messed up so use this [0] to reduce or turn off thinking, fix tool calling, keep a 100% KV cache hit rate, etc.

[0] https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates

▲

skrebbel 40 minutes ago | parent [-]

I'd love to understand this more. Are you saying the Qwen team spends their very impressive human and compute resources on publishing these amazing models and then botches the chat template with mundane bugs?

Like maybe I just misunderstand what's the hard part but wouldn't you assume that people who can put together an impressive model can also write a proper jinja chat template for it?

▲

satvikpendem 2 minutes ago | parent | next [-]

Yes for the first question. Google of all companies didn't even get it right with Gemma for a while until recently. For some reason it doesn't seem like people can actually get these templates right.

▲

Der_Einzige 38 minutes ago | parent | prev [-]

Yes yes, oh god yes. They also spread FUD in the form of terrible recommended sampler settings.

If you're using llamacpp, turn on top-n-sigma with sigma of 1, turn off top-p/top-k. You'll thank me later.

	▲	skrebbel 19 minutes ago \| parent [-]
		How is terrible settings a case of FUD?