| ▲ | brap 3 hours ago | |
> won't stop until it achieves or exhausts your prompt This is overselling it. In all implementations of this that I’ve seen, a more correct phrasing is “won’t stop until it pinky-promises it achieved your goal”. It usually relies on something like a hook that refuses to let the agent stop unless it includes a “promise sentinel”, which is basically the model having to include “I swear I’m done” in the response (usually as a Markdown comment so the user doesn’t see it). If that sentinel isn’t in the response, the hook rejects the completion and hands it back to the agent (sometimes it also restates the goal to keep it focused). It’s useful for sure, but it fails many times for obvious reasons. More robust implementations use the hook to spawn another agent that verifies the goal is achieved, but that too isn’t prefect and sometimes performs worse. Of course if you have a problem that can be deterministically verified, you could have the hook run this verification instead, but this is usually not the case (and even when it is, agents will often cheat the verification e.g. by deleting a test file). | ||
| ▲ | hungryhobbit an hour ago | parent [-] | |
The key thing is that normally Claude grades Claude's work, so you do get a"pinky promise, as all of Claude's training bias tells it to plunge ahead (even if that means "fixing" a test by commenting it out). With /goal, a different Claude instance evaluates the goal. This is the robust version you described, and such an "adversarial Claude's" approach, with or without/goal, is exactly how you avoid pinky promises (from one Claude). | ||