> Even under generous assumptions - 0.01% offender prevalence, 90% detection accuracy, and just 1% false positives - you’d correctly flag ~40,500 offenders while generating ~4.5 million false alarms. For every offender, over 110 innocents are treated as suspects.

Playing devil's advocate here, but you can skew those numbers however you want. I.e., given any classifier and corresponding confusion matrix, you can make the number of false positives arbitrarily low, at the cost of more false negatives.

We have already experience with how false positives are skewed in practice, even case goes all the way to court.

Because ostensibly good people do not want to see the CSAM material, they believe what algorithm/first reporter stated, and ofc nobody "good" wants to let a pedophile go free.

And so the algorithm tries to hang a parent for making photo of skin rash to send to doctor (happened with Google Drive scanning) or a grandparent for having a photo of their toddler grandkids playing in kiddy pool (happened in UK, computer technician happened upon the photo and reported to police, if not for lawyer insisting to actually verify the "CSAM material" the prosecution would not actually ever check what the photo was of)