...especially with megakernels...

I don't understand why megakernels haven't become the default choice. The speed advantage is crazy. I made a few changes to the openjet megakernel to run in a dual gpu (older gen, not blackwell) box and doubled my token generation speed on qwen3.8-27B (4-bit though).

I keep debating adding a third matching gpu to see if I can push the speed a bit more and move back to the 8-bit quant, but I'm stuck on the idea of just building a Blackwell gpu system despite the cost factor. Plus a third gpu is going to going to be weird for the tensor split and P2P data syncs and my workstation is old with 3rd gen PCIe etc etc.

Architectures (like Gemma4) have things like sliding window attention that make it pretty impractical to do on a GPU, you would have to do additional things to make it work, and the talent to do that, even with LLM assistance is very small.