| ▲ | vineyardmike an hour ago | |||||||||||||||||||||||||||||||
According to TM, the creators, they benchmarked it to run at 5x the speed of Qwen 27, so there is probably some issue on your end. It runs a fraction of the active parameters, even with looping, so it should be much faster. | ||||||||||||||||||||||||||||||||
| ▲ | syntaxing an hour ago | parent [-] | |||||||||||||||||||||||||||||||
This has been observed by the broad community for llama cpp. | ||||||||||||||||||||||||||||||||
| ||||||||||||||||||||||||||||||||