| ▲ | aunty_helen 4 hours ago | |
Thanks so much. 27b is a favourite model and last night my dev ran some benchmarks to achieve 133tks about 3.2x which changes the economics. | ||
| ▲ | xenonite 3 hours ago | parent [-] | |
great to hear, which quant + KV quant + KV size are you on? is it llama.cpp? | ||