Remix.run Logo
▲ TacticalCoder 31 minutes ago

> A probability score of 0.85 still means ~150 wrong answers per 1,000 checks, so plan ahead for false positives and false negatives.

But that's only true if we believe the value of 0.85 in itself is correct!? For all we know it could be 0.75 or 0.9 no?

If were to make calls to something like JEV and know that the probability the answer is correct is, say, 0.85193847098713, wouldn't that be even more amazing than JEV and LLMs themselves!?

I'm not being facetious here: how do we know it's 0.85 and neither 0.75 nor 0.9?

And if 0.85 is "correct", why stop there? Why not 0.85193847098713?

Any explanation is very welcome.