If you've got money to burn on tokens, the way that seems best to me is to set up a repeatable harness - docker container with a specific past commit from your own project, set of known issues/features that you've already fixed/completed of varying levels of complexity.

Set up a script that launches the harness for each model, prompts them to implement one of the tasks, let it churn until either tests pass or it hits some budget limit.

Then, most importantly, read the transcript and output and judge subjectively - I don't think this actually can be narrowed down to a score, although tokens burned to fix, whether it actually got the tests green etc are all good signals.

(I've done this, but so far only on a codebase that was too complicated with models that were too weak because I didn't want to spend more than a few dollars - results were inconclusive, planning on iterating on my personal benchmark in future)