| ▲ | mike-the-brain 2 hours ago | |
Might be off-topic but: is it possible to perform such a quantization on Apple devices? Something like Mac Studio Ultra M1 (even if it would take weeks/months)? | ||
| ▲ | smcleod an hour ago | parent [-] | |
Unsloth use a property dataset they don't release, however you can indeed create quantisation locally on your machine and it's pretty easy, llama.cpp comes with everything you need. | ||