Remix.run Logo
▲ gavmor 2 hours ago

Just to reiterate the question: QAT approximates quantization-errors in full-size fine-tuning, so that the model itself is insensitive to (or accommodating of, I'm not sure) them when quantized to the corresponding format.

Quantization to a non-corresponding format promises none of those accommodations.