| ▲ | danielhanchen 6 hours ago |
| We re-uploaded Gemma4 4 times - 3 times were due to 20 llama.cpp bug fixes, which we helped solve some as well. The 4th is an official Gemma chat template improvement from Google themselves, so these are out of our hands. All providers had to re-fix their uploads, so not just us. For MiniMax 2.7 - there were NaNs, but it wasn't just ours - all quant providers had it - we identified 38% of bartowski's had NaNs. Ours was 22%. We identified a fix, and have already fixed ours see https://www.reddit.com/r/LocalLLaMA/comments/1slk4di/minimax.... Bartowski has not, but is working on it. We share our investigations always. For Qwen3.5 - we shared our 7TB research artifacts showing which layers not to quantize - all provider's quants were not optimal, not broken - ssm_out and ssm_* tensors were the issue - we're now the best in terms of KLD and disk space - see https://www.reddit.com/r/LocalLLaMA/comments/1rgel19/new_qwe... On other fixes, we also fixed bugs in many OSS models like Gemma 1, Gemma 3, Llama chat template fixes, Mistral, and many more. It might seem these issues are due to us, but it's because we publicize them and tell people to update. 95% of them are not related to us, but as good open source stewards, we should update everyone. |
|
| ▲ | evilduck 5 hours ago | parent | next [-] |
| I just wanted to express gratitude to you guys, you do great work. However, it is a little annoying to have to redownload big models though and keeping up with the AI news and community sentiment is a full time job. I wish there was some mechanism somewhere (on your site or Huggingface or something) for displaying feedback or confidence in a model being "ready for general use" before kicking off 100+ GB model downloads. |
| |
| ▲ | danielhanchen 5 hours ago | parent | next [-] | | Hey thanks - yes agreed - for now we do: 1. Split metadata into shard 0 for huge models so 10B is for chat template fixes - however sometimes fixes cause a recalculation of the imatrix, which means all quants have to be re-made 2. Add HF discussion posts on each model talking about what changed, and on our Reddit and Twitter 3. Hugging Face XET now has de-duplication downloading of shards, so generally redownloading 100GB models again should be much faster - it chunks 100GB into small chunks and hashes them, and only downloads the shards which have changed | | | |
| ▲ | CamperBob2 4 hours ago | parent | prev [-] | | Best policy is to just wait a couple of weeks after a major model is released. It's frustrating to have to re-download tens or hundreds of GB every few days, but the quant producers have no choice but to release early and often if they want to maintain their reputation. Ideally the labs releasing the open models would work with Unsloth and the llama.cpp maintainers in advance to work out the bugs up front. That does sometimes happen, but not always. | | |
| ▲ | danielhanchen 3 hours ago | parent [-] | | Yep agreed at least 1 week is a good idea :) We do get early access to nearly all models, and we do find the most pressing issues sometimes. But sadly some issues are really hard to find and diagnose :( |
|
|
|
| ▲ | magicalhippo 2 hours ago | parent | prev | next [-] |
| Appreciate the work of your team very much. Though chat templates seem like they need a better solution. So many issues, seems quite fragile. |
|
| ▲ | sowbug 6 hours ago | parent | prev | next [-] |
| Please publish sha256sums of the merged GGUFs in the model descriptions. Otherwise it's hard to tell if the version we have is the latest. |
| |
| ▲ | danielhanchen 5 hours ago | parent | next [-] | | Yep we can do that probs add a table - in general be post in discussions of model pages - for eg https://huggingface.co/unsloth/MiniMax-M2.7-GGUF/discussions... HF also provides SHA256 for eg https://huggingface.co/unsloth/MiniMax-M2.7-GGUF/blob/main/U... is 92986e39a0c0b5f12c2c9b6a811dad59e3317caaf1b7ad5c7f0d7d12abc4a6e8 But agreed it's probs better to place them in a table | | |
| ▲ | sowbug 5 hours ago | parent [-] | | Thanks! I know about HF's chunk checksums, but HF doesn't publish (or possibly even know) the merged checksums. | | |
| |
| ▲ | zargon 4 hours ago | parent | prev [-] | | Why do you merge the GGUFs? The 50 GB files are more manageable (IMO) and you can verify checksums as you say. | | |
| ▲ | sowbug 2 hours ago | parent [-] | | I admit it's a habit that's probably weeks out of date. Earlier engines barfed on split GGUFs, but support is a lot better now. Frontends didn't always infer the model name correctly from the first chunk's filename, but once llama.cpp added the models.ini feature, that objection went away. The purist in me feels the 50GB chunks are a temporary artifact of Hugging Face's uploading requirements, and the authoritative model file should be the merged one. I am unable to articulate any practical reason why this matters. |
|
|
|
| ▲ | dist-epoch 5 hours ago | parent | prev [-] |
| What do you think about creating a tool which can just patch the template embedded in the .gguf file instead of forcing a re-download? The whole file hash can be checked afterwards. |
| |
| ▲ | danielhanchen 4 hours ago | parent [-] | | Sadly it's not always chat template fixes :( But yes we now split the first shard as pure metadata (10MB) for huge models - these include the chat template etc - so you only need to download that. For serious fixes, sadly we have to re-compute imatrix since the activation patterns have changed - this sadly makes the entire quant change a lot, hence you have to re-download :( |
|
