I always found that if you are good enough at whittling down boilerplate that coding becomes something akin to pure architecture.

I find that mediocre programmers and LLMs are bad at both. They're helpful if you want to shit out some repetitive boilerplate or perform a complex search of some kind but otherwise you're better off without.

Even if I was paid for using LLMs I would still find the experience nightmarish. I really don’t want to be dependent on such a system and spend my time managing fleets

How exactly will models get cheaper?

Idk, ymmv. I’ve had very good outcomes writing embedded firmware for complex systems, drivers, etc. it’s really, really good at not bodging the kinds of things that always catch meaty developers unawares.

OTOH, we run extensive harness optimization, where everything is specified in advance, then a plan is made, then a naive review of the plan vs the specification vs the blast radius, then implementation, tests, then a naive review of test coverage, a naive review of the code vis a vis our code guidelines, a review for smells, a review for silo violation and architecture compliance, a reconciliation of the documentation, then planning the next subfeature, etc. probably 10 percent code generation, 40 percent documentation and planning and adversarial review, 50 percent automated adversarial code review.

We use a Claude for planning and generation, sol for adversarial review. Our metrics say we are about 2x. Productive, at a cost of about $300 per dev per month.

We are also shipping less bugs and better, more clearly written documentation (we use technical writing English style guides implemented by Claude)

I think a big part is the constant adversarial review by a different model with no prior context except the coding standards. Also important is context management, we do an onboarding and wrap-up for each session where we have a batch of continuity documents- Learnings, musings, and roadtrips where we let the most successful high-context sessions research and then write about something that they “ became curious about” during the session. That actually brings in a lot of insight to the team and occasionally is brilliant.

Also critical is crossing compaction barriers (standard re-onboarding protocols, writing transition documents prior to compaction, etc)

Also critical is being able to smell when a session is going off the rails. What we do there is sideline the session, wait for master to advance a bit, then have it do a “4c’s” review of the committed work, blast radius analysis, and remediation. If there’s still useful context left we put it into a project unrelated to its failure context. That salvages the valuable session context without staying in the failure trench.

It’s a lot like herding cats.

For what it’s worth, I’ve made really good, state of the art software in my areas of interest using LLMs. So I do believe you can produce really good software using them. But it’s domains where I have decade of experience.

But even with that result I don’t think it’s something we should bet the whole industry on, and something I personally don’t feel comfortable relying upon