TypeSafe defines the probabilities Jev returns as "calibrated probabilities". "Probability" here means the probability of the answer being correct. If the probability is 10%, the choice should be correct about one time in ten. So, when Jev returns 83% probability it should be correct about 83 times out of 100, but the choices were only correct about 19 times out of 100, and the true probability is 1/6.

"Higher probability should correspond to a greater chance that the answer is correct."

https://docs.typesafe.ai/introduction/machine-learning-prime...