| ▲ | safety1st 2 hours ago | |
If the agent's changes are terrible, the first thing I would look at is not the agent itself, but the context: spec, scope and prompting. We generate an AI branch immediately every time a spec is completed. It then gets a brief review in our team meetings when the spec is introduced. We just take a few minutes to walk through what the AI did and take notes on what was insane and what was a good approach. #1, this informs us about the actual quality of the spec, if the model did something wrong it will often become evident that the reason was an omission, ambiguity or contradiction in the spec. #2, it saves the developer a lot of time, because in ~3 minutes we've usually established that at least half the code the model wrote was acceptable for use. Dev tasks look more like "Here's the implementation in concept, but pieces X, Y and Z will require your expertise to formulate a cleaner approach." And since AI tends to do a cleaner job at smaller scoped problems, it tends to be a great help to the dev at this stage too. If anything I feel this process increases quality, not reduces it. It certainly speeds things up. Most major challenges and problems are immediately exposed by the model's proposed implementation. We don't have to wait for the developer's first cut to discover them. They go into the task knowing exactly what to focus on, and having the uncontroversial parts of their eventual PR essentially "pre-approved." The result is where we would have had to do multiple iterations of a feature, they often come back at the very next meeting having totally nailed it. | ||
| ▲ | smashed an hour ago | parent [-] | |
I was referring to what I observe working with various teams and open source projects using AI, not necessarily best practices or what a curated workflow can produce. Even before LLMs, most teams don't bother cleaning up. The best you'll see is feature branch getting squashed on merge. AI is not really a factor here. Teams/orgs that did not care before still won't, only 10x more. | ||