I get 150 tokens/sec on my coding agent, if you are on a MacBook, thats with an M1

try kapitsa as the inference engine with pi as the harness

It works really well

Man, my M1 Pro unfortunately only has 16GB of RAM; it has been such a limiter to using local models. I'll give kapitsa a try, I see I should be able to squeeze some smaller models on