Idk, ymmv. I’ve had very good outcomes writing embedded firmware for complex systems, drivers, etc. it’s really, really good at not bodging the kinds of things that always catch meaty developers unawares.

OTOH, we run extensive harness optimization, where everything is specified in advance, then a plan is made, then a naive review of the plan vs the specification vs the blast radius, then implementation, tests, then a naive review of test coverage, a naive review of the code vis a vis our code guidelines, a review for smells, a review for silo violation and architecture compliance, a reconciliation of the documentation, then planning the next subfeature, etc. probably 10 percent code generation, 40 percent documentation and planning and adversarial review, 50 percent automated adversarial code review.

We use a Claude for planning and generation, sol for adversarial review. Our metrics say we are about 2x. Productive, at a cost of about $300 per dev per month.

We are also shipping less bugs and better, more clearly written documentation (we use technical writing English style guides implemented by Claude)

I think a big part is the constant adversarial review by a different model with no prior context except the coding standards. Also important is context management, we do an onboarding and wrap-up for each session where we have a batch of continuity documents- Learnings, musings, and roadtrips where we let the most successful high-context sessions research and then write about something that they “ became curious about” during the session. That actually brings in a lot of insight to the team and occasionally is brilliant.

Also critical is crossing compaction barriers (standard re-onboarding protocols, writing transition documents prior to compaction, etc)

Also critical is being able to smell when a session is going off the rails. What we do there is sideline the session, wait for master to advance a bit, then have it do a “4c’s” review of the committed work, blast radius analysis, and remediation. If there’s still useful context left we put it into a project unrelated to its failure context. That salvages the valuable session context without staying in the failure trench.

It’s a lot like herding cats.

Hi, this really resonated with me and the approach I have been following within enterprise software domain.

Do you mind outlining your stack around this. I know you mentioned python to support your verification harnesses but I am more interested in the agent setup. Are you specifically using Claude and it's skills with custom plugins or are you using other harnesses such as pi. I have settled on superpowers plugin across Claude and Codex, Cursor and most of my time is spent iterating through the design doc between Claude and Codex, implementing with either and starting another review cycle with the implementation, using TDD approach. It can be a lot of work but the end result is more than if I had done it myself. I am trying to formalize more. Anyway, thanks for the great insights.

How much does your whole setup cost? Just curious. An estimated range would be interesting to have