| ▲ | ilaksh 15 minutes ago | |
Not really. It's only 1B active, and he was probably using the 5 gb quant. Also he said it was over 200 tokens per second. But that still not a breakthrough or anything for such a small active set of parameters that is quantized | ||