While I don't believe they are doing the following: you can calibrate by inspecting the reasoning traces. That is the relevant distribution. If you ask someone to explain how/why they are classifying something one way v another, you can get a reasonably good understanding of their confidence level.