Remix.run Logo
kqr 2 days ago

I find the recommendation in favour of discrete categories over numeric probability for confidence level intriguing, because it goes against the grain of all advice I've read for humans interested in forecasting.

The easiest argument in favour of numeric probability is that it's simple to evaluate. Someone claiming to be 90 % certain better be right about nine out of ten times -- no fewer, but also not too often! This is useful when you're learning to judge your own confidence, which most people are bad at. (Almost everyone can train themselves to be better at i but some people seem to be good at it with no training.)

The typical argument against categories is that people can mean very different things with the same word: https://i.ibb.co/kQT4Ymz/q.png

It seems like TFA gets around these problems by creating a translation table between a fixed set of categories and the probabilities implied by the model for those categories. That's an interesting approach!