It's great to see another competitive open weight model. Congratulations!

But as is, it's probably hard to become popular amongst hobbyists and local model enthusiasts like Qwen's 27B class models or MOE models because it's a 65B dense model and inference will be prohibitively slow on consumer grade hardware, and seems like the goal/mission of the company behind this model is to automate chip design.

So if I have to assume and connect the dots, this is for the companies who want to use AI in air-gapped environments for chip design so that they protect their IPs (not leaking to third party AI labs/hostile parties) and can afford lots of server grade inference hardware for a quick unlock with the big dense model.

Neat!

▲

0x00cl 2 hours ago | parent | next [-]

Maybe is a bit more niche, but I'm pretty sure there is plenty of people who have enough RAM to run dense models like this one, I don't know how slow is slow but people are willing to sacrifice speed if it means they get a good model running locally.

Look at "SC117/Qwen3.8-Flash-Next-GSQ-RCO-abliterated-GGUF" on HuggingFace, It says it has 3.5M downloads this month and its offering a model that is 2-bit and 3-bit quantized (66GB - 83.9GB).

If this gets quantized to 4-bit (Q4_0) which usually there is very little loss in intelligence compared to the base model, then the model will probably be around 40GB, and looking that people are downloading a model that is at least 20GB bigger, then it might allow even more people to run a bigger model.

But it will only be popular among hobbyist if its as good as it claims to be.

▲

sorenjan 2 hours ago | parent | next [-]

Qwen 3.8 Flash Next is a mixture of experts model though. It has 6B active parameters per token, so you can cache large parts of the model in RAM and move it in and out of VRAM as needed. If you do that with a dense model it gets much slower.

	▲	Tade0 11 minutes ago \| parent [-]
		Experts are not typically moved between VRAM and system RAM during inference, as it would be too slow. What happens instead is that the "dense" parts of the model like e.g. the router are put into VRAM and the remainder of VRAM is occupied by the KV cache and whatever experts still fit. The rest - usually just the experts - reside in RAM, but it's bo big deal. Additionally, some models utilise an n-gram table, of which only one row is read per token, so it can actually be stored on disk without issue. Example: https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RC...

▲

stewartpark 2 hours ago | parent | prev [-]

I hear what you're saying - there are people who will use it even at 0.1 tokens/s and run tasks over night or over multiple days if it is good. but Qwen3.8 Flash Next is a MoE model (6B active) so the memory bandwidth/compute required to calculate each token is very different from a 65B (meaning, 65B all active) dense model!

▲

59nadir 4 hours ago | parent | prev | next [-]

Is it competitive overall or just competitive when compared to "U.S. Open-Weight models that are otherwise pretty decent at agentic coding"? The latter does not imply the former.

▲

regularfry 4 hours ago | parent | prev [-]

What'll be interesting is how it performs relative to Qwen3.8 27b when quantised to the same size. Which, of course, is a comparison they don't want to do.

	▲	40four 4 hours ago \| parent [-]
		Well, obviously. These newer American models aren’t competing with Qwen yet. They are competing for American businesses and organizations that can’t/won’t use Chinese models. They are playing catch-up, but I’m happy to see new players to the game, and it will be interesting to watch how they develop