I started doing it months ago and, to be honest, what the agent chooses to do isn’t unpredictable.

The problem is that different people prompt so differently.

For example, I may ask like “test different variations of this annotation on k8s pods of this service on this X cluster because it proves Y theory.”

But you know what my coworker asks? “Test Y theory.” If you were to ask two different junior engineers that, one might try random things on production and the other one might run local tests! It’s such an unguided “do anything you want as long you figure it out” request and the agent reads it like a junior who has not been told any boundaries but has been strongly told “figure it out.”