Isn't that a worry then that the same bench has so much difference in what triggered fallback for one model and what did not in another?

this "feature" is one of the primary that caused me to cancel and move to exclusively open weight based systems