Remix.run Logo
mike-the-brain 2 hours ago

Might be off-topic but: is it possible to perform such a quantization on Apple devices? Something like Mac Studio Ultra M1 (even if it would take weeks/months)?

smcleod an hour ago | parent [-]

Unsloth use a property dataset they don't release, however you can indeed create quantisation locally on your machine and it's pretty easy, llama.cpp comes with everything you need.