Remix.run Logo
CharlieDigital 15 hours ago

The point of reviews is that the process of reviews is a feedback cycle where we can identify where our docs are short. We then immediately update the docs to reflect the correction.

Over enough time, this gap closes and the need for reviews goes down. This is what I've noticed as we've continued to improve the docs: PRs have stabilized. Mid-level devs that just months ago were producing highly variant levels of quality are now coalescing on a much higher, much more consistent level of output.

There were a lot of pieces that went into this. We created a local code review skill that encodes the exact heuristics the senior reviewers would use and we ask the agent to run this in AGENTS.md. We have an MCP server over HTTP that we use to deliver the docs so we can monitor centralized telemetry.

The objective is that at some point, there will be enough docs and improved models that the need for human reviews decreases while quality of code reaches a steady state that is more consistent than any human team of varying skill level could produce.

One thing we've done is to decouple the docs from the codebase to make it easier to update the docs and immediately distribute updates orthogonal to the lifecycle of a PR.

(I'll have a post at some point that goes into some of what we are doing and the methodology.)

g-b-r 13 hours ago | parent [-]

> The objective is that at some point, there will be enough docs and improved models that the need for human reviews decreases while quality of code reaches a steady state that is more consistent than any human team of varying skill level could produce

There will never be a point when human reviews will be less needed; you're doomed to ship something horribly insecure at some point, if you ever remove them; please don't.

bmd1905 11 hours ago | parent [-]

[dead]