| ▲ | CamperBob2 4 hours ago | |
Best policy is to just wait a couple of weeks after a major model is released. It's frustrating to have to re-download tens or hundreds of GB every few days, but the quant producers have no choice but to release early and often if they want to maintain their reputation. Ideally the labs releasing the open models would work with Unsloth and the llama.cpp maintainers in advance to work out the bugs up front. That does sometimes happen, but not always. | ||
| ▲ | danielhanchen 3 hours ago | parent [-] | |
Yep agreed at least 1 week is a good idea :) We do get early access to nearly all models, and we do find the most pressing issues sometimes. But sadly some issues are really hard to find and diagnose :( | ||