Remix.run Logo
▲ xlayn 2 hours ago

Snap, no gguf to download, will wait for one to appear to test, any improvement over an already good model is good news.

"18 days on 144 H200 GPUs" insane.. didn't thought so much hardware would be needed to train Qwen.

▲drdaeman an hour ago | parent [-]

Rather than waiting, ask your assistant to quantize for you. It could be not the best quantization possible but it’s relatively quick, and I had an overall positive experience.

▲smallerize 36 minutes ago | parent [-]

It is already quantized. You need to convert to GGUF format though. Did your assistant explain this wrong?

▲drdaeman 32 minutes ago | parent [-]

My bad, I haven’t really looked yet (and haven’t asked any assistants), thought it’s BF16 safetensors as usual, and for those I used to quantize to ggufs with just a couple commands.