There are so many of us with weak GPUs and these updates are massive wins. Still waiting for the 2080 moment where this 27B param model and a reasonable kv cache can exist and function at slow, but overnight-burning speeds ..

▲

3eb7988a1663 2 hours ago | parent | next [-]

As an experiment I took, not this current quant, of Qwen 3.8 27B and had it analyze some of my writing in CPU-only mode with llamacpp. Tried a Q4 and Q5. It was dreadfully slow, but you fire that off before bed, and you can get results the next day. This was a ~6000 word markdown file, took maybe 10 minutes before it started generating anything at all, but then it just cranks away. I would need to tune down the default thinking intensity, but it otherwise did the job.

Not compatible with vibe coding, but if you want a local tool to review content, it is possible today.

	▲	jedbrooke an hour ago \| parent [-]
		I’ve been using Bonsai 2 (also based off of Qwen3.8 27B) on my mac mini in this way (using pizza-bot for the harness). In my case I’ve been using more for deep web search type stuff. It’s been a nice workflow to kick off a request in the evening and come back to a 1 page report on it the next morning.

▲

Incipient 44 minutes ago | parent | prev [-]

This would depend hugely on the price of electricity, however I'm pretty sure where I am (at roughly 25c/kwh in usd) the price of running qwen costs more than 6.1-luna per token. So other than privacy (which doesn't matter for my work) there isn't any justification for running locally. Is privacy purely what people justify local models on?

	▲	stkdump 25 minutes ago \| parent [-]
		Doesn't Qwen3.8 do much better than Luna?