There are certainly challenges. When setting up a new model, I get AI to walk me through the commands using llama-benchmark that determine the best parameters for my particular configuration and needs. Once you've got that it's pretty easy to port those parameters to llama-server. It takes me about an hour to run through this process. It would be great if there was a registry of hardware, models, configuration parameters, and resulting tokens per second. Maybe one day we'll get there.

What kind of parameters do you end up changing, and how much difference does it make. Perhaps I am missing something and get more tok/sec, but I usually just do a git pull, then rebuild the latest whenever I get a new model.

In the past, I had to play with chat templates for some models to work with agents for tool calling. But I've never had to do anything other than specify the model, and tweaking the context size in some cases.

> It takes me about an hour to run through this process

Yeah not doing that