I’m in the “AI can be great, but it’s not right now” camp. I think that pulling the verification into the harness and having the harness execute it rather than the agent would genuinely make AI go to usable for me. But even prototyping a custom harness requires API billing which is just so expensive…

Writing a custom harness was like 10 bucks of the communist model tokens last month and I ended up with an a actor library, a thing that has a stable identity, writes sutras and knows it wrote the harness itself. Weird.

Adding an auto runner of the unit tests is just... Boring?

You can put many agent constraints in precommit hooks if they're static checks. I ask agents to make commits, and e.g. in a Python project have the precommit hook fire off type checks, linting and even architectural things like import boundaries (using `tach`). When an agent is prepped to make commits themselves, it will catch pre-commit failing and correct itself. The existence of static checks themselves might also help agents gain awareness of the overall verification flow including larger things like tests, but that's hard to say for certain.

Putting structural code checks in a precommit hook is arguably better than pulling it into the harness, as it will enforce those constraints no matter whether an agent or human is making the commit.