| ▲ | sjones671 7 hours ago | |
Thanks for saying this. There's so much nonsense out there online about local models being better than Opus 4.7 and the like. It's just not true for regular users. I have a brand new M5 MacBook Pro - top end with all the specs and I've tried local models and they're barely functional. | ||
| ▲ | Yukonv 7 hours ago | parent | next [-] | |
What models and quantizations have you been trying? I've had great success with the larger Qwen 3.x models at 6-bit levels. Using 6 bit quantization is really the bare minimum to give local models a fair shot at agentic flows. Once you start pushing below that the models become more "dumb" from the limited bit space. | ||
| ▲ | SecretDreams 6 hours ago | parent | prev [-] | |
The main benefits for local are: 1) control 2) privacy 3) transparent cost model Cloud has tremendous value for speed, plug and play, and performance. You need to decide how those compete with the benefits of local - both today, and a year from now, e.g. | ||