Right... and I said exactly that in both my comments, but I was looking for confirmation that that's the issue being stared at, as it wasn't obvious to me that's what was meant and (I will cite as demonstration of the confusion) now people are bringing up other ideas (such as field order, which is an entirely unrelated constraint). (FWIW, since you are the person I was originally asking, I will take this as positive confirmation that "yes: I am saying it is better to retry the entire tool call" ;P.)

Ah, I see. IMO the other response [0] I gave is probably the closest to what I had in mind. Interestingly, retry very well might not be better, especially when measured on a pareto frontier including cost.

I have a reasonably strong prior that it should be better though. That comes from my observations that whenever I notice the difference in distribution the retry method is better (obvious sampling bias, but what kind of person ignores their own experiences). As a stronger piece of evidence and intuition, the entire goal of these systems is to reproduce a certain distribution. If that distribution says that you should use ripgrep 10x as often as you should use grep then that's a signal you probably don't want to throw away. Constrained decoding might very well do so and place them on even footing or skew 10x the other direction. Retry faithfully captures that original distribution. I'm skeptical of solutions which want to shy too far away from it (similarly with most applications of changing the model's temperature).

The more important point is just that they're different, so we should probably measure the impact.

[0] https://news.ycombinator.com/item?id=50245714