Remix.run Logo
▲ pu_pe 2 hours ago

There is always quality degradation when quantizing, but some models keep surprisingly good performance at aggressive levels of quantization too. DeepSeek v4.1 flash at 2-bit is still a beast for example. In my own experience I would easily trade off accuracy/intelligence for a vast speedup, especially if your setup simply wouldn't run something at 4-bits.

▲zobzu 13 minutes ago | parent [-]

sometimes yes sometimes no :)

that's why different models exist.