| ▲ | Qwen 3.8 27B(huggingface.co) |
| 209 points by erdaltoprak 2 hours ago | 30 comments |
| |
|
| ▲ | satvikpendem 12 minutes ago | parent | next [-] |
| As usual, the Jinja templates are messed up so use this [0] to reduce or turn off thinking, fix tool calling, keep a 100% KV cache hit rate, etc. [0] https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates |
| |
| ▲ | skrebbel 4 minutes ago | parent [-] | | I'd love to understand this more. Are you saying the Qwen team spends their very impressive human and compute resources on publishing these amazing models and then botches the chat template with mundane bugs? Like maybe I just misunderstand what's the hard part but wouldn't you assume that people who can put together an impressive model can also write a proper jinja chat template for it? | | |
| ▲ | Der_Einzige 2 minutes ago | parent [-] | | Yes yes, oh god yes. They also spread FUD in the form of terrible recommended sampler settings. If you're using llamacpp, turn on top-n-sigma with sigma of 1, turn off top-p/top-k. You'll thank me later. |
|
|
|
| ▲ | onlyrealcuzzo 2 hours ago | parent | prev | next [-] |
| If the benchmarks don't lie, this is getting very close to Opus 4.6 capability - which was the turning point for me for when AI was "good enough" that it became very hard to justify not using it. I'm sure there's some benchmaxxing going on, and some things you get only with a a larger model. But I'm feeling pretty confident if not by Gemma 5 than by mid 2028 we'll have local models that are almost always as good as Opus 4.6 was and in many cases far better. |
| |
| ▲ | DanielHB 2 hours ago | parent [-] | | What kind of things you only get with a larger model? | | |
| ▲ | redox99 12 minutes ago | parent | next [-] | | Asking it factual information[1]. You just can't compress the entire human knowledge into a 30GB file. [1] Without searching the internet. And even if you allow it, you'll get much worse results because search means browsing and parsing the top results, and search results are horrible, whereas internal knowledge from training encompasses the entire internet plus all books including very niche stuff. | |
| ▲ | onlyrealcuzzo 30 minutes ago | parent | prev | next [-] | | Similar to the way they asked Sol to solve Erdos problems, that's what I want my model to do for programming. I don't want to try to take my best educated guess at what the best design is BEFORE implementation - especially if you're designing a feature for a codebase you're not an expert in, you don't know like the back of your hand (i.e. one that is mostly or entirely LLM generated). What sounds good on paper - often times becomes unideal in practice when you get to the reality of implementation. It may not be worth re-architecting your entire system to get to a "pure" design that would be the best - all things considered. Instead, I'd like the model to independently design many plausible and coherent good solutions, then implement each of them, then intelligently pick the few winners (after its fixed any bugs that could be causing promising solutions to look artificially bad) - unless there's an obvious one - and then give me the data I need to make an informed decision on which one to go with, all before I even look at the design or implementation. You're not getting this from a one shot prompt from a 30B model today. You can't even really get it from Sol or Fable - IME. But you can get somewhat close. | | |
| ▲ | alex7o 28 minutes ago | parent [-] | | This is a harness problem not a model problem, try prime agent it can do that and it will do it well even :P but you need to prompt it in according to its tools and processes. |
| |
| ▲ | versteegen 2 hours ago | parent | prev [-] | | IME using 5.6 Luna and DS V4 Flash, I notice that although they are excellent at programming, even Opus-like in the way they try to debug, the thing they are worst at is inferring user intent and making good decisions with little information. They are absolutely terrible at that, will misinterpret small wording ambiguities. I suspect that's an ability you can't add with RL training, that it requires the depth of understanding from vast pre-training. |
|
|
|
| ▲ | xlayn 2 hours ago | parent | prev | next [-] |
| The file "Just loads" on llama.cpp, the Unsloth https://huggingface.co/unsloth/Qwen3.8-27B-GGUF is an MTP file, I see mostly the same speed on pp and generation.
There has to be something wrong with those benchmarks, I find extremely hard to believe a 27B model can work similar or exceed opus 4.6. |
| |
| ▲ | minimaltom 25 minutes ago | parent [-] | | Worth distinguishing knowledge/task benchmarks from IF / agentic. It doesn't seem out of the question that you can have a small model thats generally good at instruction following and long-horizon agentic, as usually in those cases any requisite knowledge is in the context. Most of the benchmark improvements afaict are in agentic and instruction following benchmarks. |
|
|
| ▲ | erdaltoprak 2 hours ago | parent | prev | next [-] |
| This is one of the most important model releases since most use cases don't need SOTA/Frontier If you want Qwen3.8-27B Serving Configs for the DGX Spark vLLM NVFP4 and RTX 4090 llama.cpp GGUF I added the setups here https://x.com/ErdalToprak/status/2088299678085308761?s=20 |
|
| ▲ | Almondsetat 26 minutes ago | parent | prev | next [-] |
| The $1500 Intel B70 with 32GB of VRAM can run this model at max context with good performance, btw. If you don't want to drop $5-10k for running DeepSeek this is your best budget option for local refactor/small scale dev help |
| |
| ▲ | bogzz 24 minutes ago | parent [-] | | Oh, can it work with the /v1/completions/ auto-complete endpoint? | | |
| ▲ | Almondsetat 18 minutes ago | parent [-] | | Sorry, I wrote autocompletion by force of habit. I simply meant it can complete code you have already created a structure for, which personally is very nice | | |
| ▲ | bogzz 17 minutes ago | parent [-] | | I thought so, but thanks for the clarification. I am a little bit disappointed that local autocompletion models have been left by the wayside in favor of models post-trained for agentic coding. Both Codestral and Qwen-2.5-coder are more than a year old at this point, but local auto-complete seems to me to be such a great usecase. |
|
|
|
|
| ▲ | piyh 2 hours ago | parent | prev | next [-] |
| Qwen 3.6 is ~$2/m tok, 3.8 should be drop in replacement. Gemma 31B is $0.34/m tok. The price differential on these models is massive on openrouter. |
| |
|
| ▲ | lossolo 9 minutes ago | parent | prev | next [-] |
| Why weren't the points merged again from the "dupe" thread that had 289 points? https://news.ycombinator.com/item?id=49299684 What a weird mechanism. If someone is judging a thread/topic/event impact by the number of points it got, then doing this unfairly degrades that thread. It should have deduped by user and combined the 168(at the time of writing this comment) + 289 points. Just add the twitter link from the previous thread as an additional link in the description, like you normally do, move all the points over, and remove the old thread. |
|
| ▲ | synergy20 2 hours ago | parent | prev | next [-] |
| I wish this can run directly on my RTX 4090, seems like 30B is the sweet spot for dense model to run locally, sadly RTX 5090 is very expensive and I need a new PC and new power supply(and UPS) to run that, adding a second RTX 4090 is another option, but not sure if my PC can do that yet. |
| |
| ▲ | KyleJune 8 minutes ago | parent | next [-] | | Others in this thread said it runs on RTX 4090. | |
| ▲ | baron3dl 2 hours ago | parent | prev [-] | | even a 3090 will give you the VRAM headroom. i run Q8 on an 3090/A6500 combo. well, Q8 of 3.6-27B. I'm building the Q8 GGUF for 3.8 now, assuming mine will finish before someone else's. |
|
|
| ▲ | TomGarden 2 hours ago | parent | prev | next [-] |
| Really excited to see what people do with this. 3.7 27B was probably the best compromise between size and intelligence to run on consumer hardware |
|
| ▲ | naasking 10 minutes ago | parent | prev | next [-] |
| Can anyone confirm whether this new Qwen release is any more concise when thinking? Overthinking was the biggest (only?) downside of the Qwen models. |
|
| ▲ | ThouYS 2 hours ago | parent | prev | next [-] |
| I am so happy right now, qwen3.6-27b was an absolute game changer. To see another one in the same league.. phew |
|
| ▲ | tristor 19 minutes ago | parent | prev | next [-] |
| I'm hoping to see folks distill this with current generation Opus / Fable reasoning traces. I have had my best results locally so far from Qwopus (Qwen 3.6-27B w/ Opus 4.6 reasoning distilled). This looks GREAT and I am definitely setting this up later today. |
|
| ▲ | expedited123 2 hours ago | parent | prev | next [-] |
| Kinda was expecting to see Gemma 4 26B in benchmark comparisons :( |
| |
| ▲ | kamranjon 2 hours ago | parent [-] | | Since Qwen 3.6 27b outperforms Gemma 4 26b in most benchmarks I'm not sure the value - also Gemma 26b is a MOE model whereas this is a dense model, so not typically direct competitors at their sizes - Gemma 4 31b comparison would be interesting though. | | |
|
|
| ▲ | Mr_Eri_Atlov 2 hours ago | parent | prev [-] |
| This is the homelab model hands down |