| ▲ | fuddle 2 days ago | |
I don't think SWE Verified is an ideal benchmark, as the solutions are in the training dataset. | ||
| ▲ | joshuahedlund 2 days ago | parent [-] | |
I would love for SWE Verified to put out a set of fresh but comparable problems and see how the top performing models do, to test against overfitting. | ||