The part that makes me uneasy is when the AI writes both the code and the tests. They might agree with each other and still miss the same thing. How are people here checking for that without doing the whole job twice?

We split them. The agent writing the code never writes the checks it's judged by, and those separate checks caught way more than its own tests did.

Not sure about that. This was structurally the same before AI: the author of the code wrote the tests. The tests still helped.

I like fault injection (fuzzing, chaos), and mutation testing. I think introducing these helps the AI write better tests and helps having some confidence in the tests that are written.

You should be using two (or even three) different LLMs. For example: Claude to do the coding, and Codex to do an adversarial review. This can go back and forth for several iterations before converging on an acceptably "correct" version of the code...

Occasionally, Claude may not be able to fix the issue – and it may turn out that Codex can, because it has a different approach.

Either way, at least two LLMs is the only way to get (mostly) reliable code.

[flagged]

[flagged]