| ▲ | deflator 3 hours ago | |
I second this. Anyone considering running their own hardware would also consider quantization since they would have full control over what quant to run. The value proposition changes a lot if you can get 90% of the quality for 50% the price with a quant due to halving your hardware requirement. Thank you for the article, though! Very informative. | ||