NGL: I don’t really have a good way to find out right now. It also doesn’t matter that much because the way the models use the tokes varies a lot. Qwen 3.8 is known for overthinking while Muse Glimmer may be a little slower per token, but it uses them very efficiently, caveman style.

Generation speed isn’t the bottleneck anyway, at least on pre M4/M5 devices (the newer chips got significant processing acceleration). It’s prompt processing time. OpenCode’s system prompt can take up to 3 minutes to process, which is why good prompt caching is essential.

For that I use omlx, which can persist the KV cache to disk, chunked so you can reuse parts. This helps with the usability a lot, when an agentic session is warm it runs pretty smoothly. New requests can take a couple seconds (sometimes many, which must be fixable somehow).

So: It’s not fast, but I also don’t find it awfully slow. My use is typically semi-interactive, for fully interactive use you have to wait a bit, but it’s possible. I personally am still regularly amazed that something even close to this is possible on completely local hardware.