Instead of thinking through all the places in the code the AI is going to have to touch, why not kick off three parallel agents implementing the thing and finding out what they did and the tradeoffs they found?

Planning is essential but it doesn't survive contact with reality. However, AI makes contact with reality cheap! Why not use it to improve designs, by writing the design after a few implementations have already been made?

Only slightly tongue in cheek.

▲

AnimalMuppet an hour ago | parent [-]

I'm looking for the places that need to be touched, but that wouldn't occur to me (and maybe not to an AI either) while I'm knee-deep in the code. Seeing where the AI touched isn't going to solve that.

▲

barrkel an hour ago | parent [-]

If the feature works, and passes AI auditor agents with various hats (thinking of auth and security in particular), did that code you're not thinking of need to be touched? What effect did it have that cannot be captured in side effects, tests or audits?

▲

AnimalMuppet an hour ago | parent [-]

If what you said doesn't make the AI think of changing that code, why is it going to make the AI auditor think of testing that code? That's what a gap looks like: Nobody changed it, nobody tested it, but some business constraint is now left in an inconsistent state because some piece got updated and another piece did not.

Here's an example. You updated the code that interfaced with the database. But you forgot to update the stored procedures within the database. As a result, the database is now being put in an inconsistent state with every transaction that uses your new code. That is the kind of thing that a software design doc can help you remember, because it is supposed to make you think through all the stuff.

And if you're going to say "Your business stuff shouldn't be able to get into an inconsistent state", well, there's a lot of businesses that have potential landmines laying around. You can say they shouldn't. You're right, in an ideal world. But in this world, they do, and you have to live and work in the world that we have.

Now, in fairness, a good AI check might turn up that the database was left in an inconsistent state... if it understood the constraints well enough. If. I wouldn't want to gamble my production database on the AI's understanding and testing of all the constraints, though.

	▲	barrkel 10 minutes ago \| parent [-]
		When I've worked with systems that had these kinds of characteristics, we had checklists. A long list of "have you thought of X". You can't rely on someone writing a design to think of these things either! You need to have a process, and the process applies whether you dive into the code, dive into the spec, or have an AI dive into either. It's orthogonal. To be clear, I'm not suggesting blindly deploying an AI-written spike implementation to production, but rather using it to elicit information for better designs. The fact that a probe that goes off and modifies tables X, Y and Z to achieve the feature gives information for an AI auditor to look for other uses of X, Y and Z, and discover things humans may miss, because with good guidance and a proper harness, AI is usually more persistent and thorough than people. It can turn search results into a checklist and the harness can track completion, and so on. I am far from convinced that your example would not be found via this route.