Harness and benchmark for the harness feels like a chicken and egg problem. The harness is to optimize the interaction results with the models. Any benchmark for harness has to focus on the goals that the harness was trying to optimize for unless we are only focussing on generic harnesses.

At this point when all the models have been trained on all available data with the similar algorithm,

1. either you get more data which is not feasible,

2. or get a better algorithm - a possibility ,

3. or write a more targeted harness.

Harnesses for legal, medicine and all are the ones which are getting focus for this reason. Writing benchmarks for these targeted harnesses would be a catching task