Remix.run Logo
▲ seanmcdirmid 2 hours ago

That MoE model is fast. 30B A3B gets 140 tokens/second on my M3 Max, and…the A3B part means a reasonable KV cache size and reasonable prefill times. 3.8 27B is a dense model that is mostly unusable, 40 tokens/second using MTP sure, but any cache miss is a few minutes of prefill. You are better off going with a lobotomized 3.8 flash MoE for the same amount of memory.

The report is still bizarre, but that model is still very relevant to local users.

▲rockdoe an hour ago | parent [-]

I have no idea why you wouldn't be using Qwen3.6 35B-A3B with that setup. Or did you mistakenly think that was what they're talking about?

Their model is over a year old, like, a few decades in LLM history.

▲UncleOxidant an hour ago | parent | next [-]

Or even Qwen3-Coder-Next? Still and old model, but newer and more capable than Qwen3-coder.

▲seanmcdirmid 29 minutes ago | parent | prev [-]

ah, you are exactly right. Coder and even coder-next is useless these days, you should be using 35B-A3B.