| ▲ | IChooseY0u 13 hours ago |
| This is something that Chinese models like Kimi/GLM will never care about. These kind of limitations along with the cyber-program nonsense is exactly why people will avoid OpenAI/Anthropic in the future. |
|
| ▲ | nolok 12 hours ago | parent | next [-] |
| Qwen 3.8 27B being so good, so easy to run on premise on cheap hardware and having so little limitations compared to what the US decides can or cannot be done is really eye opening. |
| |
| ▲ | stwrt 12 hours ago | parent | next [-] | | The main tradeoff I’ve found with running models like Qwen 3.8 locally is speed. On my hardware, they’re significantly slower than models served through OpenRouter, ChatGPT, or Claude. That’s been fine for smaller tasks like analyzing a document, but for more involved work like refactoring code the latency makes it harder to iterate. | | |
| ▲ | nolok 11 hours ago | parent [-] | | Yeah for live it's a bit on the slower side though still usable on the 4070 ti and 5080 our usual office desk has here, but I found it awesome running on background tasks rather than on demand, things like analyze this and that with this and that ruleset and applicable law warn me with source and justification for any error or missing part etc, qualify each returns with a ranking (certain with source, probable with reason, suggestion),... With its thinking mode enabled and it's great context size abilities it's really powerful, and each employee basically has their own assistant working in the background checking their files on its own as the day progresses (the main reason is not just cost, it's data not going out and even mess leaving the EU, this essentially frees us of a lot of hurdles) | | |
| ▲ | peri-cl 10 hours ago | parent [-] | | You find Q3 to be usable quality for your purposes? (I assume Q3 if you're squeezing it into 16 GiB) | | |
| ▲ | nolok 10 hours ago | parent [-] | | Yes, the generic system I built has unsloth/qwen3.8-27b with a 48000 context size running on each employee system with lmstudio, with a simple runner to watch folders and pass it to the llm alongside all context (our company design rules, our legal documents, our contractual rules, our applicable local law and reglementation, ...), return is then mailed to them or added to their personal dashboard event list, depending on their settings. This is clearly pushing it memory wise and the gpu offloading is only partial but LM studio deal with it automatically and it's being fine even on the 4071 Ti desks (Ryzen 7500F and 32 GB of ram), employees get any feedback in ~20 minutes after they dropped a file (it's much smoother on the 5080 desks obivously), for live it's useless but as background helper it's great and the very large context allows us to fit all the rules we want in there. One caveat has been to not ask it if everything is ok, but to find what's wrong - but always source and explain it and justify itself, never drown the user in warning in suggestions; goal is to help and provide a second pair of eyes not make them feel annoyed or unsecure. And I found people to genuinely enjoy something that works for them on their own machine and is not tracked "by the boss", thus the assistant reference, than than a centralized mothership like we also have and they have access to. I also allow them to disable it if they want, I trust them with their work, but a second pair of eyes is always great. (my previous workhorse for this was Qwen3-14b but it's missing a lot more edge cases) | | |
|
|
| |
| ▲ | 12 hours ago | parent | prev [-] | | [deleted] |
|
|
| ▲ | ValentineC 9 hours ago | parent | prev | next [-] |
| Isn't it more that the "Chinese models" are open weight? Also, the system prompt in the article is supposedly for the web version, so I think the $$$ API version or providers that still allow third-party harnesses like OpenAI should have fewer limitations. |
|
| ▲ | lubujackson 11 hours ago | parent | prev | next [-] |
| Maybe the Betamax/VHS is the best analogy for frontier vs. open models. Better tech doesn't matter if it is non-tenable for the general public. Eventually, the higher volume product will win. |
|
| ▲ | tiahura 12 hours ago | parent | prev | next [-] |
| In the future we'll all use multiple models. |
| |
| ▲ | nolok 11 hours ago | parent | next [-] | | We would be doing it now if it wasn't cutting into frontier companies' revenue. Eg if you use Claude, you probably want fable architecting, a couple of opus under it managing sub project and sonnet doing the actual function code, because fable coding a "run a query and filter the result" is a massive waste of abilities. But their own sub agent downgrade is limited to one level so if you use fable it will never direct sonnet coders. | | |
| ▲ | ValentineC 8 hours ago | parent [-] | | Claude can be told to run OpenCode or other harnesses via CLI. I've been doing this myself, all the time. (The ability to get second opinions and code reviews from Sol and other models is priceless.) There's nothing restricting it to only controlling subagents that are other Claude models like Opus and Sonnet. Also, I'm not sure that there's a one-level downgrade. Subagents can be pinned to any model: https://code.claude.com/docs/en/sub-agents |
| |
| ▲ | CuriouslyC 11 hours ago | parent | prev [-] | | I tell Astra to plan, delegate and check the work of GLM 5.3-flash and Terra medium/high subagents now. | | |
| ▲ | tiahura 8 hours ago | parent [-] | | i keep the others in tmux panes and tell the main agent to ask them for their plans and while they run in the background, formulate its plan. Then, when done, review the other plans, and revise as warranted based on other plans. |
|
|
|
| ▲ | Bluestein 13 hours ago | parent | prev [-] |
| ... along with, perhaps, model castration à la pre-embargo Fable.- |
