Remix.run Logo
a_e_k 10 hours ago

That's at BF16, so it should fit fairly well on 24GB GPUs after quantization to Q4, I'd think. (Much like the other 30B-A3B models in the family.)

I'm pretty happy about that - I was worried it'd be another 200B+.

zenmac 7 hours ago | parent [-]

are there any that would run on 16GB Apple M1?

bigyabai 7 hours ago | parent [-]

Not quite. The smallest Qwen3 A3B quants are ~12gb and use more like ~14gb depending on your context settings. You'll thrash the SSD pretty hard swapping it on a 16gb machine.