Remix.run Logo
▲ Speeding up code review with `small-change`(lethain.com)
28 points by alexmolas 4 days ago | 15 comments
▲lukeify 4 hours ago | parent | next [-]

> Something is a small-change if it meets all of these criteria it is less than 1,000 lines of code

Only my personal view but even 1000 LoC is not a "small change".

▲andreimackenzie 3 hours ago | parent | next [-]

This struck me too, but I think it is intentionally a high limit to prevent problems like relatively low impact/high LoC tests from tipping a small change into a large one. That would encourage gamification like skipping useful scenarios to keep a change "small".

The other criteria are more important.

▲skydhash 4 hours ago | parent | prev | next [-]

Yep a small changes is one computer screen of diff (36 to 48 lines) excluding generated changes (lock files, formatted files,…)

▲koolba 2 minutes ago | parent | next [-]

It can be even more lines as long as the hunks are continuous. Once you lose the context of change and have to “zoom out” to see where it fits in, the review time grows significantly.

▲jaggederest 2 hours ago | parent | prev [-]

A thousand characters, perhaps, and we can gently exclude whitespace in languages insensitive to it.

▲jibalt 2 hours ago | parent | prev [-]

AND, not OR.

▲lukeify 2 hours ago | parent [-]

How the rule is composed with other rules is irrelevant to my statement that a small change is not "1000 lines".

▲MattDamonSpace 3 hours ago | parent | prev | next [-]

A flipped bool is a single line change, or even just a single character change, and yet fundamentally changes everything about the program outcome

I do not think this is a good metric

▲superfrank 32 minutes ago | parent [-]

Despite the name, the label isn’t is only added if it meets about 5 other criteria, files and folders can be excluded meaning any PR with a change in them isn’t eligible for the label, and the PRs still need to pass all the non-AI quality checks.

Even if it’s not perfect, I think it’s the right idea. There’s some category of changes that are low risk and simple enough that a human reviewer isn’t needed and if you can identify those then you’re going to be able to save human review for the things that really need it.

▲fishtoaster 4 hours ago | parent | prev | next [-]

We arrived at a similar concept, though ours it the negation of this: we have a series of deterministic checks that can potentially add the `needs-human-review` tag to a PR. If that tag isn't present, you can merge your own PR; otherwise you need at least one human.

I think we're probably around 50/50 needing human review vs not.

Our rules for whether something needs a human review are currently mostly filepath-based: are you touching the api, or an important internal contract, or adding db migrations? Though I'd like to make it a bit smarter as we're definitely getting false positives.

▲BobbyTables2 an hour ago | parent | prev | next [-]

In my experience, the “small change” merge requests tend to be the most consequential.

Sure, they look harmless, but it’s all the external pieces (not shown in the diff) impacted by the “small change” tend to make such a disaster.

▲otiose_tortoise 2 hours ago | parent | prev | next [-]

We have something similar. I made a service that you install in your GitHub repo. It has configuration where you can set the review bar for specific paths. That review bar could be an AI agent with a specific prompt has to bless the change, so you can do stuff like "any change except to billing is okay.” If that passes you can merge your own PRs, no review needed.

Something like 30% of PRs in our company are merged like this. It's a huge velocity win when you don't have to wait around for a review.

▲demibabs 3 hours ago | parent | prev | next [-]

I feel like everyone knows at this point that the only solution to “ai code review” is to just… not have a human review the code at all.

▲beart 3 hours ago | parent [-]

Every bad change here, that has been merged in the last 6 months, correlated with a lack of human review. Sure, someone hit the approve button, but it was done within 30 seconds of the PR creation time, clearly no review was performed. This is in repos that have deterministic as well as ai driven review CI checks.

And that doesn't even get into the "features" that were implemented without reviewing the design or requirements were actually hit (they were not).

I would love to replace all of these useless meat bags with more ai. But I would absolutely still want to be in the loop to review things.

The lesson I've learned about AI is that lazy/bad engineers without AI are still lazy/bad engineers with AI.

Okay, sorry, I don't know what I'm arguing anymore, this turned into a rant.

▲0xbadcafebee 4 hours ago | parent | prev [-]

I introduced a similar change at work for Terraform changes. One big PR can create many planned changes in many different root modules; one tiny requested change in that PR can thus hold up a lot of stuff that doesn't need to be held up. So I added Claude rules/skills that instruct AI to break up the PR into the smallest possible functional/mergeable units of work, make one PR for each, and stack PRs on inter-dependencies. The result may be 40 PRs, but 38 of them can be approved and merged immediately. This lands more changes sooner, so work being started by others incorporates more current work, leading to less rework/refactoring/rebase fixes. It's Continuous Integration on autopilot. (https://martinfowler.com/articles/continuousIntegration.html)

The author is only seeing 10% as small-change, but my version has 80% as small-change. This is due to Terraform forcing you to be intentional about the borders of functional software. There is a minimum-viable-deployable component in Terraform called a Resource: you can add more resources to your module (increasing the scope), but you can't get any smaller than one resource. So it's relatively easy to define what makes a module functional, and reason about how big it gets, before deciding to break the module up. (Technically you can -target specific parts of a single resource but it's quite frowned upon)

Compare to traditional software design, where there is no absolute minimum functionality. You can make "minimum functional software" as huge or as tiny as you want. Different people design their software different ways (often unintentionally) resulting in low cohesion and a very large scope of changes required for minimum function. If you want smaller changes, you have to intentionally design your software for it.