| ▲ | smallerize 37 minutes ago | |
It is already quantized. You need to convert to GGUF format though. Did your assistant explain this wrong? | ||
| ▲ | drdaeman 32 minutes ago | parent [-] | |
My bad, I haven’t really looked yet (and haven’t asked any assistants), thought it’s BF16 safetensors as usual, and for those I used to quantize to ggufs with just a couple commands. | ||