| ▲ | Tepix an hour ago | |
Q2 quantization. Not interested. | ||
| ▲ | tcdent 43 minutes ago | parent [-] | |
All of these projects targeting low spec systems and "100 tok/s" are the same 2 bit quant without much else. Conveniently none of them include any mention of accuracy in their published numbers. 4 bit is the floor. | ||