| ▲ | andai 3 hours ago | |
>Through our experiments, we found that you should consider switching harnesses when switching models to maximize your experience. I've seen this for years, and seen several papers that studied it: the same prompt will help with one LLM and harm performance for a different LLM on the same task. And on benchmarks which test LLM x Harness, they find that LLMs tend to do way better with their "own" harness. (Or in the case of Chinese LLMs, the harness of the LLM whose training data they borrowed :) | ||