Remix.run Logo
▲ amluto 2 hours ago

It seems interesting to me only in the sense of being useless. From the horse’s mouth:

> Confidence is derived from the probabilities

https://docs.typesafe.ai/confidence

(Why is it much easier to find AI-slop websites quoting this than it is to find the actual documentation?)

My inner Bayesian would like for Jev to provide something resembling “evidence”, although I admit that one might ask Jev questions that are somewhat awkward to treat as typical Bayesian questions. If I ask “will this PR be merged”, it’s kind of strange to contemplate the probability of a PR conditioned in that PR being merged in the future. But I bet there is a way to formalize a prior-free classifier in a way that makes Bayesians and non-Bayesians happy, possibly involving actual learned probabilities and confidence levels. If you read the literature on scoring rules, you will find that classifier scores do somewhat naturally decompose into a few interpretable terms.