| ▲ | Inkling 2 Mini(thinkingmachines.ai) | ||||||||||||||||||||||||||||||||||||||||
| 297 points by jasondavies 2 hours ago | 34 comments | |||||||||||||||||||||||||||||||||||||||||
| ▲ | montroser an hour ago | parent | next [-] | ||||||||||||||||||||||||||||||||||||||||
This is bonkers good if these benchmarks hold up in real-world scenarios. To be at the level of Qwen3.8-27B with a looped transformer model is quite the flex. They just left this note on reddit: Hope that folks find this model more amenable to local usage - the sizing was largely informed by what folks on this subreddit ask for the most haha! There are two particularly interesting aspects of this model for local users: We do our training end-to-end in nvfp4, and although we haven't exhaustively benchmarked this, folks like Unsloth have noticed that our models seem to respond very well to being quantized, maintaining 90% top-1 even at at 1-bit quantization. This model is looped, which allows more depth at the same parameter count, which we particularly notice seems helpful for coding tasks. :) This is our first model targeted at local users, so let us know how this model is and anything you want to see out of future iterations! https://www.reddit.com/r/LocalLLaMA/comments/1y1diuk/thinkin... | |||||||||||||||||||||||||||||||||||||||||
| ▲ | monksy 8 minutes ago | parent | prev | next [-] | ||||||||||||||||||||||||||||||||||||||||
The GGUF version of this that can run on Ollama/Llama.cpp is: https://huggingface.co/unsloth/Inkling-2-Mini-GGUF | |||||||||||||||||||||||||||||||||||||||||
| ▲ | stillpointlab 3 minutes ago | parent | prev | next [-] | ||||||||||||||||||||||||||||||||||||||||
I'm guessing one of these local model companies will get picked by Apple sooner or later. It is just so within their business model and they have the hardware. | |||||||||||||||||||||||||||||||||||||||||
| ▲ | yread 43 minutes ago | parent | prev | next [-] | ||||||||||||||||||||||||||||||||||||||||
Recently SensaNova also released a looping model with unbelievable numbers https://www.sensetime.com/en/news/sensenova-loop-9b-release-... | |||||||||||||||||||||||||||||||||||||||||
| ▲ | 2001zhaozhao 25 minutes ago | parent | prev | next [-] | ||||||||||||||||||||||||||||||||||||||||
I think this model will be a crazy big deal, it might be the first model that can reliably run multi-agent reasoning harnesses on local hardware, as this is a MoE model and can have several agents each with their own KV caches that fit into VRAM. Dethroning Qwen 3.8 is a big deal too, and I like their section about CoT monitorability. (This model beats Qwen on benchmarks - TML is not known to benchmaxx nearly as hard as Qwen.) I guess TML really are a bunch of rationalists who know what they're doing, sort of like Anthropic. | |||||||||||||||||||||||||||||||||||||||||
| ▲ | WiSaGaN 6 minutes ago | parent | prev | next [-] | ||||||||||||||||||||||||||||||||||||||||
This is actually a big deal. I think probably by 2028 we will get a large number of daily tasks done reliably by laptop size local models. | |||||||||||||||||||||||||||||||||||||||||
| ▲ | rsolva an hour ago | parent | prev | next [-] | ||||||||||||||||||||||||||||||||||||||||
Going to test this right away on my AMD 7900XTX and see how it does compared to Qwen 3.8 27B (or DaisyRose 27B that I'm currently running). Given that Inkling 2 Mini is a MoE, it should provide way higher decode speed, but with comparable performance. Exiting! | |||||||||||||||||||||||||||||||||||||||||
| |||||||||||||||||||||||||||||||||||||||||
| ▲ | gchadwick an hour ago | parent | prev | next [-] | ||||||||||||||||||||||||||||||||||||||||
Very interesting to see them publish a looped transformer. It's been reported that OpenAI are now using looped transformers, though of course not confirmed. I think this may be the first public example of a looped transformer model good enough for real world use cases? | |||||||||||||||||||||||||||||||||||||||||
| ▲ | gok 21 minutes ago | parent | prev | next [-] | ||||||||||||||||||||||||||||||||||||||||
Very cool, and the also released a research version with variable looping support: https://huggingface.co/thinkingmachines/TML-d24-varloop-rese... Idea being you can configure at run time how much thinking you let the model do per token. | |||||||||||||||||||||||||||||||||||||||||
| ▲ | mattnewton an hour ago | parent | prev | next [-] | ||||||||||||||||||||||||||||||||||||||||
Benchmarks look good, and inkling 2 small felt better than the benchmarks suggested, so I am very excited to play with this. From a research perspective, I'm even more interested in the varlooped variant they're releasing https://thinkingmachines.ai/model-card/tml-d24-varloop-resea... I'm not aware of any other working open source models like that. | |||||||||||||||||||||||||||||||||||||||||
| |||||||||||||||||||||||||||||||||||||||||
| ▲ | xlayn 11 minutes ago | parent | prev | next [-] | ||||||||||||||||||||||||||||||||||||||||
When you are sure enough about what you are doing to include Qwen 3.8 27B in your compare.. we know you are talking... let's test drive it! | |||||||||||||||||||||||||||||||||||||||||
| ▲ | jakswa an hour ago | parent | prev | next [-] | ||||||||||||||||||||||||||||||||||||||||
The masses lying in wait for Qwen 4 cry out in joy. A savior? Could it be?! | |||||||||||||||||||||||||||||||||||||||||
| ▲ | jakswa an hour ago | parent | prev | next [-] | ||||||||||||||||||||||||||||||||||||||||
When inkling first released I tested it out and thought it was the best prose from an LLM in my quick sampling -- easy-reading and easy to understand. Big hopes I can write code with one of these models one day. | |||||||||||||||||||||||||||||||||||||||||
| |||||||||||||||||||||||||||||||||||||||||
| ▲ | neya an hour ago | parent | prev | next [-] | ||||||||||||||||||||||||||||||||||||||||
Wow, that coding demo was super impressive. Can't wait to try it out. Just need to wait for the hardware market to cool down a bit so we can run these on our own commodity hardware. | |||||||||||||||||||||||||||||||||||||||||
| ▲ | walrus01 an hour ago | parent | prev | next [-] | ||||||||||||||||||||||||||||||||||||||||
Looking at the link to quantizations it looks like unsloth has already quantized it: https://huggingface.co/thinkingmachines/Inkling-2-Mini https://huggingface.co/unsloth/Inkling-2-Mini-GGUF In terms of the active parameters and token/s on slow or less capable hardware it seems like it should be in the same general performance category as qwen 3.6 35B A3B. | |||||||||||||||||||||||||||||||||||||||||
| |||||||||||||||||||||||||||||||||||||||||
| ▲ | lenerdenator 23 minutes ago | parent | prev | next [-] | ||||||||||||||||||||||||||||||||||||||||
This looks like it could run on my M2 32GB MBP. I honestly would rather have nice, focused-purpose local models than huge "let's make AGI" datacenter-hosted models. Will have to try it tonight when I get home. | |||||||||||||||||||||||||||||||||||||||||
| ▲ | AndrewKemendo 27 minutes ago | parent | prev | next [-] | ||||||||||||||||||||||||||||||||||||||||
Can anyone running one of these locally give any sense of how good they are? | |||||||||||||||||||||||||||||||||||||||||
| ▲ | holistio an hour ago | parent | prev | next [-] | ||||||||||||||||||||||||||||||||||||||||
I'm starting to believe Gestalt is actually theirs. This is very nice. | |||||||||||||||||||||||||||||||||||||||||
| ▲ | christkv an hour ago | parent | prev | next [-] | ||||||||||||||||||||||||||||||||||||||||
I'm loving this trend. My framework desktop is looking like a better and better investment. | |||||||||||||||||||||||||||||||||||||||||
| |||||||||||||||||||||||||||||||||||||||||
| ▲ | lwarfield 32 minutes ago | parent | prev [-] | ||||||||||||||||||||||||||||||||||||||||
Omg I've been working on the same thing! I was doing this off a hunch from David Noel ng's RYS posts! | |||||||||||||||||||||||||||||||||||||||||