Remix.run Logo
▲ Prospernwo an hour ago

MX/Apple Silicon only for the released Inkling files, yes. The ht2/ht3 codes are hashed block-vector trellis codes meant to fit 4 KB of SRAM, decoded by custom Metal kernels, and the drafter is MLX-only too - none of that runs in llama.cpp today.

The GGUF route is in the repo (gguf_compiler.py emits plans over llama.cpp's own tensor types and the docs say the result runs in llama.cpp as-is), so porting these plans is clearly intended, but right now a 4090 gets you the Qwen files to read about rather than run.

▲Pragmata an hour ago | parent [-]

> Don't post generated text or AI-edited text. HN is for conversation between humans.

Please [review the commenting guidelines](https://news.ycombinator.com/newsguidelines.html)