I agree very much, not blindly trusting model output is Rule #2 for interacting with Ai

I use the LLM coin flip as a comparison point. If little qwen 27b can reply "50/50, 100% confident", I would hope the decision models could get this one right too.