Remix.run Logo
mdspan a day ago

Based on their methodology it looks like the accuracy figures (99.82% for Opus 5) are the true positive rate rather than a combined metric that factors in the false positive rate as well. They claim 1 in 10,000 but it would be nice if we had a per-model breakdown for that specific test.

DonsDiscountGas a day ago | parent [-]

Really we'd want a false discovery rate but that would depend on the prior.