Remix.run Logo
▲ hugmynutus an hour ago

*TL;DR* Promising initial results but shoddy research. OP's quant did not get 1st or 2nd prize in that math competition.

----

Gave it a quick browse to verify what my local clanker told me, it seemed to check out

I believe it is Dynamically allocating weights based on fisher information of conversation traces & For some K-quant look up table stuff it can store in an importance matrix to get better compression.

The results are complete nonsense and really don't back up any of the README claims. For Inkling-2-Mini; at 5.4 GB its KL is worse than Unsloth's UD-IQ1_S: 0.0895 vs 0.0808 on English/chat/code, and 0.287 vs 0.124 across 13 languages (e.g.: horrible). It then achieved better results by post-training the qaunt'd model on sample inputs (AFAIT).

According to anthropomorphic-clud their results for GSM8K & HumanEval are within the noise regime and vastly over stated.