| ▲ | onlyrealcuzzo 5 hours ago | |
I imagine that it's just the non-linearity of audio makes it easily apparent how much quantization really matters, and that benchmaxxing is so ingrained in LLMs that if you simply run them through benchmarks, they can do well under quantization, when in reality, they have a similar drop off that they do in audio, it just doesn't slam you in the face at every turn as obviously. | ||