There is never going to be a non-zero chance with a non-deterministic system. You can put every guard rail in place and there will always be a different way tokens are input to get bad, or subjective, tokens as output.

The findings are sick and disturbing, I hope OpenAI is not only sued for it but also that Sam Altman along with Elon, Dario and Sundar should all be held accountable in front of Congress. All of these assholes have intentionally put sexual content in their models, likely including CSAM, and so if they cannot prove that it isn't part of their training data then maybe they should be able to operate as they are today.

Where is fear mongering Dario now? He loves to drag his trope around about how advanced and dangerous his models are with respect to cyber security. Yet... We never hear him say how dangerous they could be with respect to generation of CSAM! Maybe because that wouldn't help him IPO?

> non-zero

is it ever zero? is non-zero even a problem for sane usecases?

> Dario

are you saying claude reproduces CSAM from the training set? like, in ascii?