Remix.run Logo
groby_b 12 hours ago

Here's the thing every discussion around this tries to weasel around: All else being equal, yes, more PRs is a signal of productivity.

It's not the only metric. But I'm more and more convinced that the people protesting any discussion of it are the ones who... don't ship a lot.

Of course it matters in what code base. What size PR. How many bugs. Maintenance burden. Complexity. All of that doesn't go away. But that doesn't disqualify the metric, it just points out it's not a one-dimensional problem.

And for a solo project, it's fairly easy to hold most of these variables relatively constant. Which means "volume went up" is a pretty meaningful signal in that context.

anukin 11 hours ago | parent | next [-]

Can you define what “all else” means here?

PRs or closed jira tickets can be a metric of productivity only if they add or improve the existing feature set of the product.

If a PR introduces a feature with 10 bugs in other features and I have my agent swarm fix those in 10-20 PRs in a week, my productivity and delivery have both taken a hit. If any of these features went to prod, I have lost revenue as well.

Shipping is not same as shipping correctly with minimal introduction of bugs.

mememememememo 7 hours ago | parent | prev | next [-]

Number of integration tests might be a good metric (until you announce that it is the metric then like every other metric, inc. profit, it becomes useless!)

For profit failing as a metric, see: Enron.

sarchertech 11 hours ago | parent | prev | next [-]

> All else being equal, yes, more PRs is a signal of productivity.

Yeah but all else isn’t equal, so unless you’re measuring a whole lot more than PRs it’s completely meaningless.

Even on a solo project, something as simple as I’m working with a new technology that I’m excited about is enough to drastically ramp up number of PRs.

SpicyLemonZest 7 hours ago | parent | prev [-]

The problem is that these caveats, while tolerable in some contexts, make the metric impossible to interpret for something like Claude Code which is (I agree!) a huge change in how most software is developed.

If you mostly get around on your feet, distance traveled in a day is a reasonable metric for how much exercise you got. It's true that it also matters how you walk and where you walk, but it would be pretty tedious to tell someone that a "3 mile run" is meaningless and they must track cardiovascular health directly. It's fine, it works OK for most purposes, not every metric has to be perfect.

But once you buy a car, the metric completely decouples, and no longer points towards your original fitness goals even a tiny bit. It's not that cars are useless, or that driving has a magic slowdown factor that just so happens to compensate for your increased distance travelled. The distance just doesn't have anything to do with the exercise except by a contingent link that's been broken.