▲ | a_e_k 10 hours ago | |||||||
That's at BF16, so it should fit fairly well on 24GB GPUs after quantization to Q4, I'd think. (Much like the other 30B-A3B models in the family.) I'm pretty happy about that - I was worried it'd be another 200B+. | ||||||||
▲ | zenmac 7 hours ago | parent [-] | |||||||
are there any that would run on 16GB Apple M1? | ||||||||
|