| ▲ | gavmor 2 hours ago | |
Just to reiterate the question: QAT approximates quantization-errors in full-size fine-tuning, so that the model itself is insensitive to (or accommodating of, I'm not sure) them when quantized to the corresponding format. Quantization to a non-corresponding format promises none of those accommodations. | ||