Remix.run Logo
Remix clone Hacker News
new | show | ask | jobsGithub
▲ rglynn 14 hours ago

To me, the thing that stands out about the whole state we're in here is PR review.

Yes, in an ideal world, PRs read well, are a joy to review, reflect what you discussed etc etc. We have to be real; there is only so much we can do to that end.

I'm not sure how the best teams do PR review, from my perspective it sucks. I'm talking specifically about the UX. I've always hated Github's PR page, so I typically reviewed by pulling down the branch and opening the diff with $EDITOR.

These days I think there's really no excuse for the awful UX. Linear (a company that isn't even in the domain of code review) put out a basic PR review feature[0] that is already better than what GH offers. It's simple: point a small model at the PR, group file changes together based on theme, add some commentary and sort by importance (schema changes > openapi spec).

Immediately, so much mental load has been reduced without the reviewer or the requester doing anything. This feature is pretty damn basic, and I think there are obvious next steps like generating visualisations which a dedicated product could find the time to implement.

Keen to hear others thoughts on why this is the wrong approach, or if there are tools in wide use that solve for this, or why this isnt the right problem to focus on.

0 - https://linear.app/docs/diffs#guides

▲onion2k 2 hours ago | parent | next [-]

I'm not sure how the best teams do PR review..

They don't do them. They discuss the changes as the team are doing them (software design and architecture), they automate all the things that would be nitpicks (linting, formatting, etc), and they adhere to a strict 'Do not break the build' rule so merging in something can't be disastrous (with a lot of automated checks and tests to prove that's true). They also make sure there's a robust rollback process just in case.

Once you do all that the PR process is pointless. It never catches anything useful. The team can trust one another to merge without that gate.

▲4lx87 14 hours ago | parent | prev | next [-]

Gating integration behind code review is futile. I (and many other engineers) already automated it. My agent responds to review requests and reviews as me. Company policies enforcing human code review are futile.

I think all these platforms chasing code review are doomed. My LLM doesn't need any of this tooling.

We should be reviewing the actual working software. Systems that make it easy and instant to demo any proposed change are what is needed. Code (and specs) are going to fade into obscurity. PR review has already shifted towards validating the product (working software) over the process (code).

The future of software production is more like Replit – not GitHub.

▲dmurvihill 11 hours ago | parent | next [-]

Wow, good to know. Can you tell me what company you work for so I can make sure it's not part of my stack?

▲satvikpendem 11 hours ago | parent [-]

It's many if not most companies now doing automated PR review.

▲20k 10 hours ago | parent [-]

Asserted confidently without a shred of proof

▲gabrieledarrigo 12 hours ago | parent | prev | next [-]

> Code (and specs) are going to fade into obscurity

I'm always skeptical when I read absolute statements like this.

Especially about specs. They drive and document how the software should work, behave, and under which conditions. In the same moment you write a prompt in natural language, you are writing a spec.

Why should they disappear?

For the same reasoning, we could no longer need books or manuals or documents (is it valuable to keep around that doc about F-22 specs?).

▲abernard1 8 hours ago | parent [-]

Agreed. Specs are more important than ever. Constraints: types, contracts, protocols, schemas, APIs and such are what ground the boundaries of systems.

Without them, code drift is a guarantee.

Does anyone trust that a set of human sentences typed into a box is more reliable and precise than objectively exact descriptions of data?

This is not to diminish the utility of these code factories. But surely, if anything provides resistance to drift, it's codebase consistency, standards, and things which must objectively be true for the software to function.

▲jeremyjh 12 hours ago | parent | prev | next [-]

This entire article is about the fact that the codebase deteriorates over time when you work this way. If they are correct, it will take longer and longer to develop PRs, and more and more cycles to QA them. Eventually the code will be unsalvageable and it might get there before anyone notices a problem. I don’t know if they are right but I know codebases can get there with humans at the helm - and the humans can adapt and learn.

This is not a certain fate but it sure looks likely and is a serious risk. Some software is so low stakes that you can get away with it I’m sure - there are always trade offs.

▲satvikpendem 11 hours ago | parent [-]

The bet is that models get better faster than code gets worse. Looks like so far it's correct, as Fable can already write better code than me and most others on average.

▲dhorthy 11 hours ago | parent [-]

the thesis of the post is that this is not true. fable can solve more problems but for complex systems it is not sufficiently diligent that you can turn the lights off.

▲satvikpendem 11 hours ago | parent [-]

They turned the lights off in 2025. The post is simply not accurate to today, they even say so, stating that apparently no one has proof of better models in 2026 improving codebases over time. I've seen it happen but the author hasn't apparently, so it's a game of "you said I said", not anything concrete.

▲dhorthy 11 hours ago | parent [-]

(op here btw) - the question isn't "can models make code better" - its "left fully unattended, will they turn your codebase to slop over time"

from the footnotes (sorry if this got a little buried)

> yes of course you can get gpt-5.5 xhigh to do BRILLIANT refactors. But you had to tell it to do that. And to tell it to do that you had to understand your codebase well enough to know it needed doing. We're here talking about why lights-off wont work

i suppose this would also be a good place to ask if you fall in the "high stakes production systems" camp

> If you love vibe coding, please, go on vibing. I still vibe code lots of things, I just also maintain lots of production software (and through HumanLayer, help 1000s of other engineers do the same), so the rest of this is aimed at folks solving hard problems in complex codebases.

▲satvikpendem 10 hours ago | parent [-]

GPT 5.6 Sol and Fable are similar step changes, not 5.5, but that's beside the point. If in the Markdown instruction files you ask them to periodically clean up the code or refactor where necessary then that's what they'll do. They don't explicitly do it because otherwise people, as they had for previous models, complain where models are asked to change one thing then they rewrite whole systems, so now the latest models are explicitly trained on instruction following.

In other words you can make the AI act like a software engineer complete with fixing and improving code autonomously (Cursor gave a glimpse of this [0]). It is not a failure of the model to do this automatically but a failure of the human having the wrong instructions.

[0] https://news.ycombinator.com/item?id=48982535

▲dhorthy 10 hours ago | parent [-]

i guess to clarify my contention:

generic prompting like "review this code" or "make the architecture better" will raise the floor but cannot come close to human-quality code without humans understanding what code exists and what to ask for.

Obviously cursor and other labs have a stake in this going one direction, but I don't buy it, and I don't think you should either.

In fact, "Rebuild sqlite from spec" has all the problems with every other benchmark that I cited - model knows the whole problem up front and never has to iterate on the pile of slop it created cheating its way to a solution.

In any case, politely, I think we're mostly arguing vibes here and I'm not sure its going to get anywhere.

▲monkpit 4 hours ago | parent [-]

And what percentage of the software in the world needs to come close to human-quality code? I’d argue the percentage of the whole is VERY low.

▲regularfry an hour ago | parent [-]

The whole point is that if you don't maintain high quality, any software will degrade to the point where the factory breaks. "Human-quality" here isn't quite the right measure because human software factories also bog down in exactly this way.

▲20k 14 hours ago | parent | prev | next [-]

>I (and many other engineers) already automated it. My agent responds to review requests and reviews as me. Company policies enforcing human code review are futile.

This is also known as being a terrible engineer. If a company enforces human review and someone deliberately tries to circumvent this with an LLM, I'd fire that person in an instant

The reason for human code review is:

1. So *you* understand what's going on, not the LLM, and people can ask you questions about it

2. Because LLMs are not that good at code review

It seems weird to brag about literally not doing your job. It sounds like you could be replaced with a python script, what value do you bring?

▲dhorthy 13 hours ago | parent | next [-]

blake smith has a really good post on this - that mental alignment among the team is the primary purpose of code review - https://blakesmith.me/2015/02/09/code-review-essentials-for-...

▲4lx87 13 hours ago | parent [-]

That is a good post. Thank you for sharing.

I can't help but notice that this person is speaking as a programmer, not the person in charge of employing programmers. Is that what the business values code review for? In my experience, managers enforce code review as a quality control mechanism: the "Find Bugs" step in the article's pyramid. And LLMs are already better than human reviewers at the "Find Bugs" part. Design and alignment with PMs and sales can happen before and after software delivery. At least that's how my employer sees it (and I suspect most other businesses).

It's a rough time to be a programmer who cares about the code, who understands software is a creative process, who understands that designing the software is intertwined with the code, who cares about systems. The hyper-focus on delivery was already a lot to deal with before LLMs.

▲RamblingCTO an hour ago | parent | next [-]

I can tell you that, because I'm that person.

What I care about with my department:

- alignment on new stuff being introduced into code bases (the bigger it is, the more important to consider promixity on things the teams are working on tho). also: maybe finding a more optimal solution together, implies a proper PR process (which I always created with my teams, it takes time and we plan for that)

- code quality assessment: maintainability, regression tests, taste, security etc. etc.

- code is improved by expecting a review in the end. if you know you're not reviewed, you might rush and push out suboptimal code. hurts the baseline

Maintainability, bugs, incidents, team understanding, complexity (all impacting future delivery, uptime, user retention, onboarding etc.) are all metrics I care about as they impact the software and the company as a whole directly. A lot of CTOs or other execs/managers might not care, but there are a lot of bullshitters out there. Or maybe I'm just used to complex software in regulated/hard fields, I don't know.

▲DaiPlusPlus 12 hours ago | parent | prev [-]

> In my experience, managers enforce code review as a quality control mechanism: the "Find Bugs" step in the article's pyramid.

Nope.

To reduce it down to a single word, I'd say we do code-reviews to assess "taste".

-----

Code-review is one part of a larger process (the SDLC!), and while review does help with "finding bugs" it is not the singular reason why we do it; and other parts of the SDLC are concerned with finding bugs in the first place, namely the various Test and QA steps in the process; when your code is in the (peer) code-review step then there's an expectation that the author already identified and resolved actual bugs/defects (i.e. where actual-program-behaviour deviates from the spec[1]), so a PR for new functionality is expected to include unit and integration tests to demonstrate that.

Now Claude will gladly take a Jira ticket, write-up a plan/spec, write tests, implement the feature, verify the tests pass, address static-analysis issues, push branch, and submit the PR - and if-the-program-works then it's "correct" and so surely there's nothing really to review and so merging the changes should be a breeze... but I find myself rejecting these PRs all the time because these agents still "just don't get it"[2]. (But I'm sure they'll "get it" eventually; you can't stop progress). We can revisit this topic when we get there; but for now I'm going to reject an AI-authored PR that eschews it.

Good taste is important.

[1] I'm not going to pretend anyone actually writes any kind of spec (informal or otherwise) for the vast majority of software out there; but an unwritten-spec exists when you mentally combine a vague Jira ticket, platform-conventions and common-sense (and that's how Claude in an Agentic Loop works too, except it always has to write-out the Spec.md/Plan.md to disk first, whereas us humans keep things in our head).

[2] I cannot define "it".

▲jpollock 5 hours ago | parent | prev | next [-]

There are also compliance reasons.

SOX requires code review by a second party. If the reviews are automated, the development process isn't in compliance and that can get expensive in a hurry.

▲dboreham 4 hours ago | parent [-]

And of course these SOX-mandated second reviews were 99% of the time pure theater.

▲bandrami 2 hours ago | parent [-]

Sort of; it's like the itemizer systems at airport security. They're entirely theater (in the sense of "for show") but that show is itself important in that it makes entire categories of attacks significantly less feasible.

Sort of like how simply having a uniformed police officer on a street corner reduces street crime significantly even if the officer never interacts with anyone.

▲amannm 13 hours ago | parent | prev | next [-]

It seems weird to get angry about a change that is happening and will continue to happen due to what the market demands from software, which I believe will be the speed/ability to solve problems, rather than its own stewardship

▲20k 10 hours ago | parent | next [-]

I'm just here to do good software engineering and make good products. If people want to try and circumvent that process to deliberately make the product worse and slow development down, they're bad software engineers, its as simple as that

Nobody thinks its a good idea to fully automate PRs, at least not anyone with software engineering experience. It slows down your ability to solve problems if your code base is an absolute disaster

Again: If someone fully automates everything, what service do they provide? Why are they even there? Especially actively lying in the process when people are trying to force knowledge dissemination with human reviewed PRs

▲ 2 hours ago | parent [-]
[deleted]
▲computably 12 hours ago | parent | prev [-]

Sure, if you're shipping a 500MB mobile/web app with a 5s start-up, it probably won't make a difference today, or tomorrow, or 3 months later. The problem is when your codebase is a dumping ground for AI slop and a year from now nobody can make any changes.

Also, it's not just some subjective value put in human review. If a change causes a major outage and it turns out it's because 5 reviewers were all shirking the company-mandated responsibility of review, and it could trivially have been caught, I would 100% expect those people to get fired. It's not a process issue if people are intentionally circumventing the process.

▲satvikpendem 11 hours ago | parent | prev | next [-]

Or that their company does not in fact enforce human PR reviews or even encourages AI PR reviews.

▲0xblacklight 13 hours ago | parent | prev [-]

this is exactly right

▲NateEag 3 hours ago | parent | prev | next [-]

> We should be reviewing the actual working software. Systems that make it easy and instant to demo any proposed change are what is needed.

"Program testing can be used to show the presence of bugs, but never to show their absence!" -- Edsger W. Dijkstra

▲RamblingCTO an hour ago | parent [-]

And it doesn't take into account maintainability, complexity, security etc. I think the commentator above you is not a SWE. No one could be that ignorant

▲tripleee 14 hours ago | parent | prev | next [-]

What's your job as an "engineer" in this post-automated world?

QA?

▲roncesvalles 12 hours ago | parent | next [-]

The role of a software engineer is foremost about the ownership of software systems, and secondarily about modifying them or developing new ones.

Software development is a continuous operational undertaking, like having a garden, and not a one-shot "task" like designing a logo. Systems need to be tended to and somebody needs to know how they work, especially if they're business-critical or (in the case of software companies), the business itself.

A corollary point is that many of the rules and policies of the business are solely embedded within the software itself. There is rarely a complete set of documents that says "this is how everything works" -- if you want to know how something works, you ask a software engineer, they take a look inside the codebase and then tell you how it works. In that sense, all software engineers are also product managers (or more accurately, product managers are just crude software engineers, but that's besides the point).

▲jdzikowski 12 hours ago | parent [-]

> The role of a software engineer is foremost about the ownership of software systems, and secondarily about modifying them or developing new ones.

Yes! That's sooo accurate. Tools you use change, the level of abstraction you operate changes, but the role remains.

▲4lx87 13 hours ago | parent | prev [-]

Producing features and fixing bugs, same as it was before. The organizational process of software development has not changed much with AI: execs decide direction and initiatives, PMs decide what to build, which is broken down into features and bug fixes that SWEs produce. In my experience organizations don't actually care how SWEs produce features, except insofar as it relates to how many and how fast the features can be pumped out. Organizations see code review as a process to prevent bugs. Humans are not as good as LLMs at reviewing code for bugs. Abstract notions of code style and quality that programmers care about is not why organizations enforce code review.

▲necovek 12 hours ago | parent [-]

Before LLMs, "organizations" have seen code review as multiple things:

  * Share knowledge about a particular area of the codebase between multiple people
  * Share overall engineering knowledge between the parties in the review
  * Ensure maintainability of the codebase long-term
  * Ensure readability of the code
  * Catch architectural/directional omissions (yes, from the planning/architecture phase) before it's really too late and non-reversible change goes in (eg. large destructive DB schema change)
  * Ensure changes are small, self-contained, and as often as possible, reversible
  * Do some basic manual QA
  * Do comprehensive integration testing with a fully built-out system
  ...
  * And yes, catch bugs before they hit production.
A lot of the above could be fun and engaging, and especially knowledge sharing and ensuring maintainability/readability was a very motivating thing for me as a more experienced engineer having learned so much from getting good reviews when I was less experienced.

Programmers care about style to ensure readability and thus maintainability of the code, but also to keep changes minimal — if every diff converted between tabs and spaces in the entire codebase, it'd be impossible to see what has really changed with the simple tooling we generally use (one could build diffing tools that ignore changes like these, and they even exist, but are not ubiquitous).

▲layer8 12 hours ago | parent | prev | next [-]

> We should be reviewing the actual working software. Systems that make it easy and instant to demo any proposed change are what is needed.

You’re arguing for experiment over logical proof or reasoning. That’s qualitatively a very different thing, and the former is no adequate replacement for the latter.

▲latentsea 7 hours ago | parent | prev | next [-]

Entropy is not your friend.

▲majormajor 7 hours ago | parent | prev [-]

> We should be reviewing the actual working software. Systems that make it easy and instant to demo any proposed change are what is needed. Code (and specs) are going to fade into obscurity. PR review has already shifted towards validating the product (working software) over the process (code).

Ok, sure, if your software can be easily exhaustively re-tested on every change.

But in that case, why were you ever doing PR's in the first place? Cargo cult?

▲tcoff91 14 hours ago | parent | prev | next [-]

PRs sucked to review long before Agents were a thing, but now it really sucks because there are more to review.

▲viccis an hour ago | parent | next [-]

Like you said, they sucked back then but it was more rare to get PRs of many thousand lines to review multiple times a day in my experience. But I also run into a problem now in which I have coworkers in two camps. There are some that, when they send me a PR and I have serious concerns about it, it's because they disagree with me fundamentally on some concept and we need to chat and discuss it. And then there's some coworkers who send me with a PR I have serious concerns about because it's very clear they never reviewed the LLM output before making a PR. Those people also tend to take my comments, run them against their LLM session, and slack me back some Claude list-of-questions stuff, and often also respond to me with LLMese.

I firmly believe there's no reason to employ the latter group. I have several coworkers like this that I could replace with a Claude session or two every week and I'd still have to push back on work and review a lot, but I'd still wind up saving time because then I'd talk directly to Claude.

So I guess my point is that I dislike AI era code review because for some subset of it, I basically can't even assume a human has ever looked at it, and that by doing so, I'm doing the work for one of my coworkers who is probably doing his grocery shopping at 2PM as I read it.

▲sevenseacat 18 minutes ago | parent [-]

I received two 15,000-line PRs the other day, from the same person.

I am currently ignoring them.

▲dboreham 4 hours ago | parent | prev [-]

Right on. In many ways AI has just exposed how bogus and delusional our industry's "process" has been for years.

▲xorcist 14 hours ago | parent | prev | next [-]

> group file changes together based on theme, add some commentary

Isn't that what commits are? Or ... should be?

▲lozenge 12 hours ago | parent [-]

No, commits always display the files in a fixed order, and then display changes from line 1 to line N. An AI could select the order to display in, add per hunk commentary and automatically adjust how much context lines are displayed.

▲xorcist 11 hours ago | parent [-]

But if you need to split the commit in hunks, and add a commentary per hunk, isn't that just a sign that you really should just split your commits?

That's the most killer feature of git, that it's so easy to slice your commits any way you desire, and then redo again. The use case of taking a chunk and commit separately is so common it even got a special mode in the add command. That, and the super fast jumping between branches is what set it apart from contemporary version control systems.

The extra context provided by the review tool is gone when the review is done anyway. Review systems come and go, but the commit log is for eternity.

▲danpalmer 8 hours ago | parent | prev | next [-]

PR review sucks because engineers often don't optimise their code authoring for it. When they do it's a joy.

PR review of agent generated code sucks because agents are, in my experience, very poor at authoring for review. This makes sense because the review process is not represented in the resulting code artifact which is what they're trained on. Agents are always producing much larger changes than should be in a single step, and often touch unrelated code with poor judgement of whether it should be included or not.

▲preg_match 3 hours ago | parent | next [-]

One thing that really bothers me about agents is they generate excessively long and detailed comments.

This sounds good for review. Except… the comments are never intent or trade-offs, they’re always the “what”, which the code then duplicates right after. Comments should be short, concise “why”. Often I look at code and think “hm this is odd, I wouldn’t have done it this way, is there some constraint I don’t know about?”

The comment should explain that, not what the code is doing. If I wanted to know what the code is doing I’d read the code!

▲latentsea 7 hours ago | parent | prev | next [-]

Agents are also controllable to a degree. How agents author PRs is now an engineering problem.

▲danpalmer 4 hours ago | parent [-]

But this is my point, they're not really controllable in this way. There are no fixed rules (like tests or linters) for small PRs, it's not good enough to do it by line count.

A good PR tells a story to the reader that leaves them confident in the result. That means knowing the reader and how they think, and that is something LLMs are very bad at.

▲latentsea 3 hours ago | parent | next [-]

And you can't think of a way to bring an engineers mindset to solve that?

If you leave it up to raw LLM outputs you're not going to get the results you're after. If you put on your engineering hat and think about your problem statement and what tools you can bring to bear on it you'll find you can likely craft a PR authoring skill and a set of evals for that skill that satisfies your requirements with a high enough degree of reliability.

▲danpalmer 2 hours ago | parent [-]

I'm not coming at this cold, I've worked on the skill for PR (CL) creation in the Google monorepo. I'm not talking about raw LLM outputs here. It's a hard problem that is fairly fundamental to the training material being used for LLMs today.

I'm sure we can improve things, I'm sure it will get better over time, but I'd say we're a small fraction of the way towards where we need to be, and trending in the wrong direction with long trajectory development creating bigger changes on average.

▲dhorthy 4 hours ago | parent | prev [-]

yes well said

▲dhorthy 7 hours ago | parent | prev [-]

exactly. a great pr is a joy to review. we've found some success in agents generating static HTML walkthroughs that order the diffs in something other than GitHub's default alphanumeric ordering, but it can only go so far

▲boron1006 14 hours ago | parent | prev | next [-]

I just dont think LLMs are very good at judging importance or summarizing code.

I tried experimenting with what is ultimately a treesitter based approach - https://github.com/0x007BA7/codebook

And really liked it. Definitely nowhere near production ready but I think theres room for a player to come in and do something similar.

▲dhorthy 13 hours ago | parent [-]

yeah that was another thing i hoped would pour through here - that deterministic systems are much better for evaluating quality (test, linters, cyclomatic complexity, etc) - but that we don't have such a system for code maintainability, at least not one that's widely accepted or adopted

▲2001zhaozhao 14 hours ago | parent | prev | next [-]

I think the right shape is to review and merge directly from the agent window.

▲0xblacklight 13 hours ago | parent [-]

reviewing code as it's being written & re-steering it > reviewing it once 20k lines have been written

▲2001zhaozhao 12 hours ago | parent [-]

Yep i aim to split my work into tickets that are <=1k lines of code each. In the end it all gets squashed into a mega 10k commit regardless but i can keep track of the quality much more that way.

(more precisely the sub-tickets merge into a bigger feature branch, which in turn gets squash merged into the main branch at the end. i have a tool that automates this flow with a gui and ticket management.)

▲WorldMaker 14 hours ago | parent | prev | next [-]

> I'm not sure how the best teams do PR review, from my perspective it sucks. I'm talking specifically about the UX. I've always hated Github's PR page, so I typically reviewed by pulling down the branch and opening the diff with $EDITOR.

When $EDITOR = VSCode there's a shortcut on the GitHub PR page: if you type a . it opens in github.dev in a VSCode instance.

▲claytonjy 13 hours ago | parent | prev [-]

thanks for pointing out the linear PR stuff, hadn’t seen that. Interesting that while a dozen other companies are trying to muscle in on the hosting/versioning side of github, rather fewer are working on the PR side.