How does this make the model useless? It finds and fixes the security hole. It can even write a test that verifies that the fix didn't break things. But it deliberately doesn't reveal the fact that it was a security issue that was fixed.

Seems useful to me. But more useful for defenders than attackers.

Imagine that you have the repo A, ask the model to "fix the security issue" and end up with A'.

Just take the Diff A' - A to see the security hole.