That would make the model useless

How does this make the model useless? It finds and fixes the security hole. It can even write a test that verifies that the fix didn't break things. But it deliberately doesn't reveal the fact that it was a security issue that was fixed.

Seems useful to me. But more useful for defenders than attackers.