> Reading and Debugging 1.5 1.0 > Code Reviews 0.75 0.75

Since these numbers are made up, I may as well throw my personal anecdote in the ring. I find reading and reviewing far harder with coworkers who are using AI. Tickets contain about 5x as much meaningless junk as they used to, and testing notes - while far more thorough - are often now multiple pages in length. Reviews also contain much more code, people try to do more drive-by fixes because the models can generate those fixes so quickly, and people understand the code they're submitting far less clearly because the model is able to generate fixes they simply couldn't previously.

I feel less productive than I was a year ago, and I don't see my team shipping more features than they were previously. But everyone reports that they're far more productive. I don't get it.

▲

Supermancho 4 days ago | parent | next [-]

The numbers are completely made up. Jr developer 2.5 vs 1.0 while "regular" is 1.0 and 1.0? The more senior the bigger the work. It's the same across both, worst case.

0.75 to 0.75? Rework from review is also much faster. Now you don't have to tell a peer to rework a bit here and there for obvious reasons and spend time on a new loop. The review process isn't atomic.

Our production pipeline is faster across our very large organization, after implementing AI processes.

> Tickets contain about 5x as much meaningless junk as they used to

This is a process problem. Developers should be able to answer questions about their PRs, or you reject it. It's not a daunting blanket issue.

▲

K0balt 4 days ago | parent | prev | next [-]

Your process engineering is lacking. Just throwing AI at existing workflows seldom produces good results. Processes have to be reengineered to benefit from the strengths and cover for the weaknesses of AI systems, with observability at the right inflection points being fundamental to success.

▲

tempfile 4 days ago | parent [-]

Care to give an example?

	▲	K0balt 4 days ago \| parent [-]
		We run extensive harness optimization, where everything is specified in advance, then a plan is made, then a naive review of the plan vs the specification vs the blast radius, then implementation, tests, then a naive review of test coverage, a naive review of the code vis a vis our code guidelines, a review for smells, a review for silo violation and architecture compliance, a reconciliation of the documentation, then planning the next subfeature, etc. probably 10 percent code generation, 40 percent documentation and planning and adversarial review, 50 percent automated adversarial code review. We use a Claude for planning and generation, sol for adversarial review. Our metrics say we are about 2x. Productive, at a cost of about $300 per dev per month. We are also shipping less bugs and better, more clearly written documentation (we use technical writing English style guides implemented by Claude) I think a big part is the constant adversarial review by a different model with no prior context except the coding standards. Also important is context management, we do an onboarding and wrap-up for each session where we have a batch of continuity documents- Learnings, musings, and roadtrips where we let the most successful high-context sessions research and then write about something that they “ became curious about” during the session. That actually brings in a lot of insight to the team and occasionally is brilliant. Also critical is crossing compaction barriers (standard re-onboarding protocols, writing transition documents prior to compaction, etc) Also critical is being able to smell when a session is going off the rails. What we do there is sideline the session, wait for master to advance a bit, then have it do a “4c’s” review of the committed work, blast radius analysis, and remediation. If there’s still useful context left we put it into a project unrelated to its failure context. That salvages the valuable session context without staying in the failure trench. It’s a lot like herding cats.

▲

Rury 4 days ago | parent | prev [-]

You don't get it? It's like this:

"Make me a picture of a house".

> AI proceeds to draw a house.

"No that's not right, it should be a red bricked house. Not a brown one."

> AI redraws a red bricked house.

"No. It should have a 2 car garage, sit on top of a hill. Also it should have a front porch, and have a tree right in front."

> AI then draws a red bricked house on top a hill with a 2 car garage with a front porch and a tree right in front.

...

The issue is that people think AI should automagically create some vague idea in of theirs, without having to do the work of spelling out all the exact details. So AI (like people) must make some assumptions about what wasn't specified. Like since you didn't specify a "red bricked house" in your initial prompt, but merely a "house"... it had to come up with something as to the color, and did as you otherwise asked, but it didn't know you actually wanted a "red bricked house", since you never specified that detail. Hence why it failed to do what you wanted, and you had to "review and correct it".

Again this isn't solely an intelligence problem, but an inherent problem in language/communication, of unsaid assumptions/specifications. Of being unaware of what you don't know, unaware of your own assumptions, sometimes even being unaware of what you even want. It's why AI can't fully get rid of jobs in software.

But sometimes people just don't care. They just want a picture of a house made. Anything remotely resembling a house will do, not necessarily a solid one, or one that can withstand a magnitude 8 earthquake. You know... like something people can put up in minutes so they don't have to do hard work... such as a shabby old tent. And AI is very good at generalizing, so it can in fact achieve this. So everyone reports they are far more productive now, putting up tent after tent. Meanwhile, the people responsible for the slop have a nightmare to review...