| ▲ | IronWolve 2 hours ago | |||||||
Only mx version? | ||||||||
| ▲ | Prospernwo an hour ago | parent [-] | |||||||
MX/Apple Silicon only for the released Inkling files, yes. The ht2/ht3 codes are hashed block-vector trellis codes meant to fit 4 KB of SRAM, decoded by custom Metal kernels, and the drafter is MLX-only too - none of that runs in llama.cpp today. The GGUF route is in the repo (gguf_compiler.py emits plans over llama.cpp's own tensor types and the docs say the result runs in llama.cpp as-is), so porting these plans is clearly intended, but right now a 4090 gets you the Qwen files to read about rather than run. | ||||||||
| ||||||||