> I can recognize within seconds if a PR is an easy merge or a 5 minute phone call

I could, before LLM-authored PRs. They broke all my heuristics for what to check. LLMs can write code that looks incredibly professional and well thought out, and nowadays they almost never make the usual human mistakes (like off-by-one errors). But at the same time, they can make incredibly silly mistakes of the kind people would never make (like inventing brand-new parameters for some remote API call).

It feels like everything I’ve learned over the years about reviewing code no longer applies.

I used to be able to say the same but 10k loc diffs at my org shattered this ability pretty quickly.

> I can recognize within seconds if a PR is an easy merge or a 5 minute phone call.

I'm glad you can.

Even before agents, I'm pulled to work on at least a couple different projects a year, the successful ones with teams of 10s of people working on them. If I stay longer than a couple of months, I eventually become the expert on some of the things I did, so I get pulled years later to review something I haven't touched in ages, which has changed requirements, and grown a bunch of dependents.

That was the reality of working on a team with hundreds of people and dozens of related products.

Now agents are helping people increase complexity and velocity N-fold.

You can call it a problem in the coal mine, but there's no going back. GitHub simply has no choice but to handle this new scale.

The human experience of reviewing PRs hasn't really moved beyond "look at these 2000 lines of text and think really hard".

At least for me. Am I missing something?

Code review is a problem only because it is a backstop for everything else. We skip architecture review, we skip prototyping, we one-shot a result in isolation from our team and then expect a human at the end of the line to do our job for us??

Code review is great if the code isn't a surprise, the architecture was already agreed on, the AI reviewers are already happy, and you just need a final pass.

Code review was already terrible before AI if you were slopping down weeks of work on a poor teammate to sift through.

I tell the younger engineers at my company the same thing today I said 5 and 10 years ago. Make the review easy for another engineer to do it, and they'll do it. Make it hard on them, and you won't ship shit.

I don’t believe that nobody is doing it well. May I ask who is doing it well? Either in house or with an AI reviewer vendor.

For me, what works is highly templated directory structure and rules that linters can quickly check and enforce. Every PR is a cookie cutter design filled in. There are only a handful of things you need and they’re all just CRUD at the end of the day. You can employ the same very few design patterns and primitives across a lot of your stack. Review is so fast, if you trust your CI to do its job and the developer to actually QA their own work.

What isn’t working is the sheer volume of PRs. Even with CODEOWNERS directing attention to only high impact changes, it is a constant distraction. I’m not sure how to handle SOC2 where we need a human reviewer on all code. Lastly, agents shit the bed when they touch spaghetti, esp when it’s sitting next to structured patterns. IME, the best way to get around this is to just rewrite features from scratch. But in this current world of instant completion and reward, it’s been challenging to convince people to commit to more ambitious lifts.

Finally, I want review more than ever. A team can’t keep up with the pace of change even if it was all perfect. I like the idea of more up front planning, but the code is ultimately what matters.

We're trying to solve this with Theseus. Would love to hear your thoughts! https://theseus.dev

> A PR centric cycle is like being in a factory and standing next to a conveyor belt of PRs. Humans can't possibly review properly.

Then something is either wrong with your conception of PRs or whatever process you have been submitted to. PR is asking for feedback from people that already understand the software. It's also a learning place for anyone that is not yet familiar with the part being modified. So you have the author, the reviewers, and the learners. Which is three separate sets of people.

The PR review itself shouldn't take a lot of time (10mn is already a lot), because it's either a small change which is easily understood[0] or it's a more complex one [1] that should have a design discussion thread attached. This discussion is orthogonal to the code itself. Both should have a description good enough for the reviewer to have an idea why the change is necessary before looking at the code.

So a PR is an exchange of ideas around a real artifact, the code. If no idea is being exchanged, then it's merely a gatekeeping activity.

[0] https://cgit.freebsd.org/src/commit/?id=73b1453e6d87f9af4c47...

[1] https://cgit.freebsd.org/src/commit/?id=49a3cbc47f8e721a787e...

So why bother with PRs at all, might as well just skip the charade and push to master.

Yeah I used Gerrit years ago then moved companies and ended up with GitHub and then GitLab. Night and day (or day and night in this case).

Hi, not all organisations are the same. I do not operate my team this way. I am sorry that this is your experience but please don’t pretend that it’s universal.

I have been thinking about a similar “semantic diff” approach to code reviews for the past several weeks. I’d love to hear more about how you’re approaching this - there’s a handful of distinct scenarios I see frequently in PRs that I think should be reviewable in a more direct form (renames, moving blocks of code between files without changing the text, whitespace changes, etc - basically any action a modern IDE can do deterministically).

I’ve played with some bookmarklets that modify the GitHub PR ui and attempt highlight these situations, but my setup is brittle (just trying to identify basic patterns in the text diff, ex “these are [oldname] replaced with [newname]”, which is not necessarily the same thing as an IDE rename-this-function action). I’ve been wondering if the answer here is in doing a diff on a compiled intermediate form of the program, visualized in some way - this would have to be language-specific though, I guess, and is well outside my expertise.

Separate from the diff aspect of reviews, I would also appreciate some more control in a PR tool over how to layout a review (what files/hunks to be shown, in what order). My team attempts to do this with carefully organized commits, so that reviewers step through them one by one, but the GitHub UI leaves a lot to be desired here (files appear in a static order top to bottom; comments and discussions can’t be stuck to one commit and show up in other commits; general bugs in the web view). I would appreciate being able to control how others see my PR if there’s room for that in your product.

Anyways, I joined your waitlist and am curious to see what you come up with - I think there are improvements to that can be made to code reviewing (that don’t involve AI - everyone is already putting AI review in their PRs), and am willing to try anything that takes a swing at that.

>1. Hunk checkpoints

How long are your files that you can't easily "finish" a file?

Y'all are still reviewing code?

(Semi serious here.)