Remix.run Logo
▲ walrus01 a day ago

Looking at the link to quantizations it looks like unsloth has already quantized it:

https://huggingface.co/thinkingmachines/Inkling-2-Mini

https://huggingface.co/unsloth/Inkling-2-Mini-GGUF

In terms of the active parameters and token/s on slow or less capable hardware it seems like it should be in the same general performance category as qwen 3.6 35B A3B.

▲jakswa a day ago | parent [-]

might need this if trying to test on llama.cpp https://github.com/ggml-org/llama.cpp/pull/32695 -- "Adds support for https://huggingface.co/thinkingmachines/Inkling-2-Mini - uses a looped architecture and this PR adds looped support"

▲hrncn a day ago | parent | next [-]

Let's hope these make it into master, and don't end up like the diffusiongemma patchsets. It is always a bit sad if the exotic and exciting models don't make it into master and all the people using prebuilt llama can't just give them a try. Tho I can understand why these in particular might not be wanted in master.

▲jakswa a day ago | parent | prev [-]

and this lol: https://github.com/ggml-org/llama.cpp/pull/32694