Remix.run Logo
nater5000 3 hours ago

I have a similar setup to the OP (although I'm not sure if I'd call it a "software factory"), and I utilize a local model for some aspects of the setup. Specifically, I have a single RTX 3090 Ti with 24GB VRAM on my (main) home server, and I've been primarily running Qwen 3.6 35B A3B out of it via Ollama (I just switched to Qwen 3.8 27B, though, and I've also tried other Qwen models as well as Gemma models).

The Qwen models are decent, but they don't come close to the full Claude experience I've come to expect. As such, I only use the local models for specific tasks where it makes sense to do so. Really the setup is that my Claude-powered agents are able to incorporate my local model into work it builds out. The agents can perform inference against the Ollama API as they see fit, and I encourage them to do so for tasks where (a) the low-level capacity of the local models make sense and/or (b) where costs can become a concern.

It seems to work well when it comes into play (like having Claude drive a web browsing session but letting Qwen handle much of the actual browser interactions, image analysis, etc.). Still, Qwen just isn't smart enough (or fast enough on my machine) to handle anything agentic that isn't non-trivial.