| ▲ | pu_pe 2 hours ago | |
There is always quality degradation when quantizing, but some models keep surprisingly good performance at aggressive levels of quantization too. DeepSeek v4.1 flash at 2-bit is still a beast for example. In my own experience I would easily trade off accuracy/intelligence for a vast speedup, especially if your setup simply wouldn't run something at 4-bits. | ||
| ▲ | zobzu 13 minutes ago | parent [-] | |
sometimes yes sometimes no :) that's why different models exist. | ||