i don't think this is doable fairly. as they say in the blog post, the only fair way to is to look for new, previously undiscovered zero-days, otherwise you always risk the model has in some way been trained on the vulnerabilities. looking for legit new stuff is the only way to prevent leakage (even accidental one)

That’s my point too.