> I would have expected there to be a hundred entities running every single model over major code like PHP, and clang, and pretty much any behemoth open source project the world relies on. This is the sort of thing the model makers should be doing as a matter of course, when they’re testing their new models.

Well, that would be super nice of them.

From a Machiavellian standpoint it may benefit the people with the most powerful models to let some serious RCEs like this PHP one get widely exploited in damaging ways to further their regulatory capture argument that open weight models have to be legally banned and only locked down models with enforced guardrails around usage can be allowed.

> From a Machiavellian standpoint

From a marketing standpoint, having the first model to find a bug of this magnitude and severity would be a pretty damn impressive proof of capability. Finding previously unknown, critical vulnerabilities in widely used open-source software could become a gold-standard test of frontier AI competence.