| ▲ | Catloafdev 2 hours ago |
| Curious how people feel about this compared to DS4 Flash, given they are pretty close in size. Also curious how well it holds up to heavy quantization. DS4 Flash can currently run reasonably well on systems with ~96gb+ RAM, I wonder if Hy3 can compete there. |
|
| ▲ | tarruda 30 minutes ago | parent | next [-] |
| > given they are pretty close in size One thing that might not be obvious about about DSV4 is how much innovation the Deepseek team implemented in its architecture. When llama.cpp fully supports its lightning indexer, the full 1M context will only require about 6G of RAM. So even though they are similar in size, I believe Deepseek will be much more efficient in that regard. > I wonder if Hy3 can compete there Highly depends on how well Hy3 is resilient to quantization. DSV4 is useful even at 2-bit quants. |
|
| ▲ | UncleOxidant an hour ago | parent | prev | next [-] |
| That's a 2-bit quant of DS4 flash. You're probably better off running Qwen3.6-27B at Q8. |
| |
| ▲ | spmurrayzzz an hour ago | parent | next [-] | | I think its good advice to test both on your own evals for sure, but the MoE parameters are already natively FP4 in ds4. Dropping to 2bpw isn't as big of a loss as it seems (and as corroborated by antirez's work). Its also only 13B active, so your decode speed would be nearly 2x that of Qwen3.6-27B. So there are other latent benefits as well. | | | |
| ▲ | Catloafdev an hour ago | parent | prev | next [-] | | For most coding or agentic tasks, Qwen 3.6 27B likely outperforms, yes. For 'general intelligence', DS4 Flash seems to be a noticeable step up still. | |
| ▲ | sosodev an hour ago | parent | prev | next [-] | | I suspect it would depend on the task. DS4-flash does, as previously mentioned, handle quantization very well. Even at 2-bit it's still very coherent. | |
| ▲ | ckocagil 40 minutes ago | parent | prev [-] | | Isn't Q8 way overkill these days? I see many graphs showing Q4 or Q5 having less than %1 deviation. Nvidia's NVFP4 Qwen quantization should be even better due to its better training methods. | | |
| ▲ | mdgld 38 minutes ago | parent [-] | | It depends on model size I think, but yeah, from my understanding at ~30B and below Q6 or even Q4 will get you 95%+ of the way there |
|
|
|
| ▲ | wolttam an hour ago | parent | prev | next [-] |
| Hy3 lacks the DSv4 architecture's KV Cache efficiency. Whereas I can run DSv4 Flash on a pair of DGX Sparks and have enough memory left over for 3M tokens of KV cache, with Hy3 (quantized to FP4), there is only room for ~130K tokens of KV cache. |
| |
| ▲ | ignoramous an hour ago | parent [-] | | Lower context window notwithstanding, Hy3's coding benchmarks hold their own against DeepSeek v4 Pro & MiMo v2.5 Pro. That's quite something for a model priced like DeepSeek v4 Flash & MiMo v2.5 (for non-cached tokens), which are 3x cheaper than their respective Pro variants. | | |
| ▲ | wolttam an hour ago | parent [-] | | It's impressive indeed. I would also expect the next checkpoint of DSv4 Flash to come in somewhere at this level (DeepSeek has had over 2 months to continue training since it released). It's exciting that the open models continue to get better and more efficient across the board! |
|
|
|
| ▲ | nunodonato an hour ago | parent | prev [-] |
| DS4-Flash is not only "significantly" smaller, it will also benefit from a lot more speed thanks to DSpark |
| |
