The title is a little misleading.

It was Opus 4.6 (the model). You could discover this with some other coding agent harness.

The other thing that bugs me and frankly I don't have the time to try it out myself, is that they did not compare to see if the same bug would have been found with GPT 5.4 or perhaps even an open source model.

Without that, and for the reasons I posted above, while I am sure this is not the intention, the post reads like an ad for claude code.

▲

mtlynch 16 hours ago | parent | next [-]

OP here.

I don't understand this critique. Carlini did use Claude Code directly. Claude Code used the Claude Opus 4.6 model, but I don't know why you'd consider it inaccurate to say Claude Code found it.

GPT 5.4 might be capable of finding it as well, but the article never made any claims about whether non-Anthropic models could find it.

If I wrote about achieving 10k QPS with a Go server, is the article misleading unless I enumerate every other technology that could have achieved the same thing?

	▲	eichin 3 hours ago \| parent [-]
		Also, he did compare with earlier versions that, before 4.5, were dramatically worse at finding the same problems. There's even a graph. That seems to pretty solidly support the idea that this is "gain of function" as it were...

▲

mgraczyk 17 hours ago | parent | prev | next [-]

No the title is correct and you are misreading or didn't read. It was found with Claude code, that's the quote. This isn't a model eval, it's an Anthropic employee talking about Claude code. So comparing to other models isn't a thing to reasonably expect.

▲

weird-eye-issue 15 hours ago | parent | prev [-]

> You could discover this with some other coding agent harness.

And surely that would be relevant if they were using a different harness.