You need to make the rule concrete somehow. I call it a "control". So for example, instead of instructing it "always run tests before committing", you (or you have it) make a git commit hook that always runs the tests first and that refuses the commit if they don't pass.

In this case, it is an advisory control only, because the LLM can also unhook that hook. And of course, it could also just disable the failing test(s) with some bullshit reason. But it is far better than assuming it will comply every time.

Then there is the "hard control", which is the inviolable that the LLM cannot bypass.

You need to move as much as is technically possible to either hard or (failing that) advisory controls.

>In this case, it is an advisory control only, because the LLM can also unhook that hook

you can in principle make a hook in the harness itself that will run an auditor prompt that checks it adhered to the policy, correct the model and also make it known in advance.

I've seen pretty much every model including Claude bypass commit hooks constantly if there's even a tiny bit of friction, instead of pausing and asking me to help fix why it can't run the hooks (usually something that needs to be done outside the sandbox like `npm ci`).

That's why it's critical to have these checks run in a context that the model can't bypass, such as via github actions that block PR merges. Generally works pretty well for stuff at work (where we have all this stuff set up as part of continuous integration checks), but it means that for personal stuff I have to set up quite a bit more infrastructure to ensure that models don't just skip running tests (which they LOVE to do).

So you are essentially saying "you can't, you can only safeguard from effects of LLM eventually ignoring it"