What kind of parameters do you end up changing, and how much difference does it make. Perhaps I am missing something and get more tok/sec, but I usually just do a git pull, then rebuild the latest whenever I get a new model.

In the past, I had to play with chat templates for some models to work with agents for tool calling. But I've never had to do anything other than specify the model, and tweaking the context size in some cases.