Where did you get these instructions?

They have a demo repo with a setup.sh script:

The release tag and weight file you suggest doesn’t match what they wrote.

I figured them out, starting from the GGUF on Hugging Face.

If you have found better instructions and they work then use those instead!

Personally I prefer to download models directly rather than running some `./setup.sh` script where I need to then review what it does first.

	▲	refibrillator 2 hours ago \| parent [-]
		Yeah just wanted to mention in case it explains the 2x lower throughout you are seeing on M5. To be fair their documentation is a bit inconsistent in some spots. Would be good to know if the release and weights from their demo repo work better. I’m trying on a 4090 and will report back.