I’ll look into adding harnesses. The benchmark sources usually don’t contain harness so the problem is limited data. What models would you like to see?

▲

bozdemir 4 hours ago | parent | next [-]

For example you might add Strata & Qwen 3.8 Flash Next with some quantizations. Nowadays it is a go to selection and very sought after MoE model.

	▲	albinstman 3 hours ago \| parent [-]
		Qwen 3.8 flash next is already in the works and should be live soon.

▲

cyanydeez 4 hours ago | parent | prev [-]

He's exagerating, I hope. The only bottleneck a harness attachs is pre-context fill and whatever tooling is installed. A proper harness reduces these to negligable startup costs.

Unless of course he's driving some weird lambo that dumps 200k of context into their chats. Who knows, I feel like we're on a yellow submarine.

▲

bozdemir 4 hours ago | parent [-]

Exagerating? I setup llm endpoints on corporate level in a company on H200 gpus, that serves hundreds of software engineers, I have observed differences, even on different versions of nvidia drivers. I shared my day to day experience. It is more and more important if you have very limited VRAM like 24gb on a 4090 RTX desktop or RTX 5090 Laptop gpu. I set up and use llms on my personal devices also, when I can't use cloud models on classified company materials. Here is an example problem what I daily report and deal with.

https://github.com/ggml-org/llama.cpp/issues/27330

▲

jimnotgym 4 hours ago | parent [-]

I can believe different drivers make a difference, but that wasn't the point under debate

	▲	bozdemir 3 hours ago \| parent [-]
		[flagged]