The referenced https://blog.exe.dev/replace-your-ci is resonating with me.

I'm spending a relatively large amount of time waiting for CI to finish running, then clicking a button for an agent to fix failures, then waiting for CI to finish again. Might as well push that down into the core development loop.

Do folks have advice for doing this in practice?

Hooks in the harness? Pre-commit hooks? Different GitHub configurations?

That blog post didn’t make a lot of sense to me. It’s saying to run CI in the merge queue. And also have CI be a script you can run locally. But also to replace CI?

Like:

> A merge queue is a script you run to push to origin/main (instead of using a PR UI like we did back in the GitHub, all-human days).

This really doesn’t capture what a merge queue is. It’s a queue of change sets to merge, and CI has to pass on each change set before it can merge. There’s still a CI process. You can’t really enforce that just by having a bash script agents run locally, you need something at a higher level to enforce it, which is… CI.

CI doesn’t have to be slow, and the author isn’t even arguing for removing the tests.

1. Make your tests as fast as practicable

2. Split out "syntactically correct" fast checks like linters into a standalone script and call it along with the slower checks in a full-check script

3. Set the fast-check script as a pre-commit hook and the full-check script as a pre-push hook.

4. Give the model instructions that it needs to run the fast-check script after every change and the full-check script when it thinks it's done.

5. Run full-check in CI.

Then you're good as long as the model doesn't bypass the pre-push hook, and even then CI will catch it.

if you're using claude code, i find you can instruct it to watch the pr and iterate on ci failures. I put this in an /issue skill that completes GitHub issues in the way I like then PR's or boots dev with QA instructions if it needs a human touch/eye. to drive this, I have a higher level bash script that makes a fresh worktree off of origin/main named after the issue and spawns a claude session running /issue.

i'm not at the point where I'd go totally hands-off with code review / QA, so this setup is the right balance of automation for my current read on agent capability.

we have pre-commit hooks for fast tests (under a minute on a 2024 mac), a playwright script (smoke tests) that can be run on-demand locally, including setup and cleanup (~5 minutes). And on top of this we also have CI tests pre-merge (which we may drop soon)

just tell it to watch CI and fix errors?