I find some of the most interesting, and catastrophic failures in my agent fine-tuning come from the clamping down of non-determinism. It is totally the correct approach, but must be handled delicately. The non-deterministic core remains, but now under bimodal pressure.

I think this is less about clamping down on non determinism and more remembering that a script is much more reliable than having the agent do some things. Think making a number of API requests to get info for context or running a sequence of testing steps to generate a report. Remove easy places where that non determinism rears its head and there is really no need. I talk about what I'm doing with PR review in a other comment, as an example.

In other words, are there places where a one liner for the agent would be more reliable than markdown instructions and crossing fingers?

I look at it this way... I wrote scripts over the years to make my life easier. Do the same for your agents and free their attention for the parts that matter.