Shame they are not including old crap like my RX580 which is doing some impressive stuff with Qwen 9B Models.

dotancohen 4 hours ago | parent | next [-]

Tell us more about what your RX580 can do with the small Qwen models. What workflows are you doing, and what kind of performance are you seeing?

▲

hbogert 3 hours ago | parent [-]

yeah opencode just runs with it with lmstudio as llm engine.

It won't win benchmarks of course, but 20 tps generation is okay and letting it chug away on my local kubernetes cluster to troubleshoot is pretty nice. First start is pretty slow though, parsing is around 100-200 tps. Which is crap for long questions or harnesses like opencode which start around with thousands of tokens. Once cache is warm, pretty enjoyable.

I think it could have way more use cases where it's just 'good enough'. Email categorization, voice commands for the home. 1 year ago this was still a very miserable experience, but qwen 9B + better cache quantization and more tricks in the latest llm engines makes it viable.

Will it do difficult architectural refactors by just giving a one-shot prompt like opus allows, nope, not gonna happen.

	▲	dotancohen 12 minutes ago \| parent [-]
		Thank you

▲

internet_points 4 hours ago | parent | prev [-]

that's 8G vram? you're using it for coding?

	▲	hbogert 3 hours ago \| parent [-]
		Yep, it's 8GB vram from 2017 :D Yeah simple, but menial refactors where IDEs don't have clear cut refactors yet. Or languages like CUE which has awesome tooling, but not so great refactors in its lsp yet.