Where's the human baseline for the benchmarks?

Also, you said:

> The best performing model scores a 66.8. So does this prove LLMs can “understand” the code? I would argue that the answer to this is “no”.

How did you reach that conclusion?

I've considered setting up a human test as a starting point. But as a hobbyist that would require volunteers which I didn't think I'd be able to get for this project. Perhaps version 2 of the benchmark could do this if there is enough interest.

> How did you reach that conclusion?

Based on the fact that the task set so far includes fairly simple goals and none of the models scored 100%.