Makes sense why they released an entire study yesterday discrediting SWE-bench Pro.

And they'd be right, it's an almost saturated benchmark where even some subpar open source models score very well on. And most models are clustered within a small range so it really doesn't tell you much.