Supposedly this is a Kimi k3 rival. Bit of a chonker, especially since they only released bf16 and fp8. So at launch this will be harder to serve than k3. No QAT on q4 means that someone with deep pockets (nvda?) will have to quant it, with plenty of calibration data. Should bring it ~1.3TB, so around k3 size.

License pretty similar to k3 with some caveats. Free to use for internal or <50M$ revenue / year. Limitations above that threshold for serving the model or services targeting coding / productivity agents.

Benchmarks are looking good, trading blows w/ opus4.8 and sol, generally 10-20p under fable. But that's neither here nor there w/ qwen, their benchmark to real world usage correlation has been iffy in the past.

The local model 3.8-27B announced for Friday, same time so ~48 hours from now. That'll be a bit more exciting for a lot more people, since 3.6 was quite good for local inference, and their 3.7-max -> 3.8-max shows a lot of improvement.

▲

ZeroCool2u 39 minutes ago | parent | next [-]

Unsloth already has a guide for their quants: https://unsloth.ai/docs/models/qwen3.8

	▲	codedokode 13 minutes ago \| parent [-]
		I wonder who is unsloth and where they got time, hardware and knowledge to quantize them?

▲

MrDrMcCoy an hour ago | parent | prev | next [-]

Llama.cpp can quantize without special training, but I'm not sure if any special model architecture support is needed to read it in the first place. If it can be converted to gguf at all and you know what tensors to target, it can get the full ternary bonsai treatment today.

▲

NitpickLawyer 37 minutes ago | parent | next [-]

Sure, but that's for "personal" serving. I meant for 3rd party providers. Usually we get a good indication on what it costs to host this, as the prices settle on open router. That's why I said it's tougher to serve than kimi k3 on launch. As a provider you'd do fp8 if the model creator didn't do QAT on q4, or until someone does a good calibrated nvfp4. And that's usually nvda :)

▲

MrDrMcCoy 27 minutes ago | parent [-]

That makes sense, but your specific phrasing precluded the possibility of non-QAT quantization.

	▲	NitpickLawyer 22 minutes ago \| parent [-]
		Should have worded that better, my bad.

▲

binary132 an hour ago | parent | prev [-]

QAT is an optimizing quantization algorithm, not naive quant.

	▲	MrDrMcCoy 41 minutes ago \| parent [-]
		Right, but the way they phrased it suggested that without QAT it could not be quanted at all.

▲

esafak an hour ago | parent | prev [-]

Now that they have reached the frontier in raw performance, I would like to see Chinese models improve their reasoning efficiency.

	▲	jingpostmedia an hour ago \| parent [-]
		[flagged]