Remix.run Logo
▲ maxgee 40 minutes ago

Our account was recently banned from OpenAI for a CSAM violation. We had an autonomous agent using an image classifier switch to OpenAI (instead of a local model it should have been using) which was classifying user-uploaded content. After classifying over 100,000 images, OpenAI banned our account.

We asked for further information so we could report the uploader to the NCMEC and remove the content from our systems. They refused to give us any information about this incident, denied our appeal, and have so far, illegally refused our privacy requests for data about our account.

OpenAI does not care about fighting child sexual abuse, this is completely performative.

EDIT: These image uploads are up to 20+ years old, and are being served through Cloudflare with their CSAM scanning product enabled. We've received no notification of any problems from Cloudflare. Something on OpenAI's system thinks that an image we sent to them for classification is CSAM, and they aren't willing to elaborate or communicate about it at all. For reference, our website is https://ytmnd.com

▲uptrails 12 minutes ago | parent | next [-]

I've done some work in this space and it's mostly that Big Tech has regulators and parent advocacy groups already very angry with them and breathing down their necks, so they would rather lose customers than take a nuanced approach. The marginal value of having you as a customer is approximately zero, the regulatory risk of letting CSAM through is huge.

Another complicating factor is that it really blows to have humans look at stuff like that, so everyone would rather use automated classifiers; and if you assume good faith and share too much info with the affected parties, this will be exploited by the bad guys to gain insights into how the system works and how to circumvent it.

I don't think there's an elegant solution to this, other than "don't allow tech companies to get so big that they can afford not to care about individual customers".

▲icantevenhold 33 minutes ago | parent | prev | next [-]

I would posit almost no one really cares about fighting child sexual abuse; most of them just use this as pretence for more surveillance and data collection.

▲hingler36 26 minutes ago | parent | next [-]

I would posit the exact opposite: it's really important to fight child sex abuse, which is why it's so frustrating that people are co-opting that cause to promote surveillance and data collection.

▲exolymph 20 minutes ago | parent | next [-]

Saying it's important is not the opposite of saying that it's rare to actually care.

▲demibabs 21 minutes ago | parent | prev [-]

How else do you prevent it tho?

▲nonethewiser 29 minutes ago | parent | prev | next [-]

[flagged]

▲graemep 27 minutes ago | parent [-]

I think the good faith interpretation of GP's comment is that almost no one among big tech companies (and maybe governments) cares.

▲nonethewiser 21 minutes ago | parent [-]

I think the good faith interpretation of his comment is that most people and companies dont care, which Im responding to. If the commenter doesn’t agree with his own words he should clarify.

I think virtually every person and company does care. In fact I think its exceedingly obvious.

Even if OpenAI, Facebook, etc dont care be cause they are evil, they do care because pf liability.

None of this even excludes the possibility that CSAM controls are co-opted for power sometimes. Which is why it’s absurd to suggest these companies don’t care - you can concede that point and still argue its being co opted.

▲grvdrm 27 minutes ago | parent | prev [-]

I agree. Because if they do they hold some sort of view on what is or isn’t in an image/video. I think it is simpler and easier to put your hands over your eyes and look the other way.

▲nonethewiser 31 minutes ago | parent | prev | next [-]

> OpenAI does not care about fighting child sexual abuse, this is completely performative.

That doesn’t really follow from your experience.

OpenAI could care about child sexual abuse and not respond to your appeal, even wrongfully.

▲ezfe 14 minutes ago | parent | next [-]

But part of that was trying to identify which content was flagged so that the uploader could be identified, but since it sounds like they didn't identify the content now the OP cannot report it to the authorities.

▲maxgee 12 minutes ago | parent | prev | next [-]

If they were serious about stopping CSAM they'd spend 10 seconds looking at the prompt we used, the emails we've sent and the fact we're trying to get an NCMEC case number and report the uploader, but we can't because we literally don't know what image they're referencing. If a human was involved in any part of this process or decision, it isn't apparent because all we've received is automated responses.

Frankly I am not interested in retaining an OpenAI account anymore so the only reason we appealed was to try to get more information, not to restore the account.

▲charcircuit 11 minutes ago | parent | prev [-]

His point is that even if they do care, they are being performative in going completely silent, doing "nothing" as opposed to working together with others to remove content and report bad actors.

▲mattmaroon 29 minutes ago | parent | prev | next [-]

I’m sorry that happened to you, but I would read this more as they are an organization not prepared for this. Why would they be? One minute you’re trying to make computers pass the turning test, the next minute child porn is getting involved.

The fact that they did not previously have proper procedures in place for something like this is not proof that they don’t care. They are one of the fastest growing companies of all time, it’s hard to get disparate parts of an organization aligned on something. Their organization has more than doubled in size in the last nine months.

▲antiloper 30 minutes ago | parent | prev | next [-]

I mean, you're holding it wrong. You uploaded CSAM to OpenAI which is presumably against their TOS.

▲ 26 minutes ago | parent [-]
[deleted]
▲teiferer 31 minutes ago | parent | prev | next [-]

Seems like they want to play in the big boys club (Google, Facebook, ... ) where this kind of treatment is standard.

▲ajjahs 33 minutes ago | parent | prev [-]

[dead]