Personally I find these kinds of things fascinating. Partially for the practical problem it represents and it's facets but also the one sided backlash of it all. Moderation is an interesting conglomerate of so many factors.

First of all I don't think Meta is a good or honest company, but that doesn't mean they're not trying at all. In reality even if they were good and honest I genuinely believe their task is probably nearly impossible. Humans just suck too much. It certainly doesn't help that Meta has at least some perverse incentive to be permissive with ads, as they get money from them. Though Meta execs aren't stupid. Inflammatory political ads are one thing, kiddie porn is another I suspect they lose more money over CSAM than all the ad revenue from them. I believe them when they say they removed over 36 million pieces of child sexual exploitation content last year.

Which is a number that I should let sink in. Basically more than one piece of CSAM shows up in their ad stream every second of every day all year. If their moderation techniques are 99% effective that results in 360,000 CSAM ads leaking to the public annually.

Another interesting thought is how much would it cost to actually get that rate to 100%? let's assume human moderators aren't paid much but are infallible. That all-in a single moderator employee costs $65K , that's salary, benefits, HR ratio, psychological therapy, equipment, the works, all factored in.

let's say on average you get about 1 ad review ~every 2 minutes, so bout 250 ads per-day per-employee factoring in some time off you actually get about 65,000 reviews a year. So bout $1 per review. So the question begs how many distinct ads enter their system each year? which I haven't found a stat for. But assuming only 1 in 1000 ads are CSAM, which still feels high but is probably a reasonable ratio. we get 36 billion ads needing review annually. At a dollar each 23% of their gross annual ad revenue each year would be just moderation overhead and they would need to hire over 550,000 moderators.

Evil or not I think they organically built a thing that operates at an unimaginable scale, that even moderation with failures in the parts-per-million would result in tens of thousands of unsavory or unscrupulous ads reaching users and they have neither the incentive nor the capacity to reach even that level of accuracy.

▲

happymellon 2 days ago | parent | next [-]

Disagree with essentially all of this.

To get a Facebook account they require all sorts of PII.

Why not have people give their PII to set up an advertising account and if they distribute CSAM then they are banned. Removing access to WhatsApp will stop a large portion of the world from participating.

And the parts that can't? Well, they can't advertise on Facebook/Instagram anymore.

▲

ticulatedspline 2 days ago | parent [-]

Probably a case of "hate the game not the player"

CSAM is highly illegal, no doubt these accounts are getting banned but likely not charged (who knows what jurisdiction the advertiser is even in) clearly they're able to pass the current bar.

This is conjecture but I'd bet that in all likelihood someone at some meeting at Meta has floated the idea of moving impedance to the signup to try to limit this phenomena. Maybe more verification, or more hoops. And likely Meta crunched the numbers and was looking at billions of dollars lost in legit ad traffic and that just doesn't fly for a publicly traded company.

I'd guess that the folks at Meta would absolutely love CSAM to stop showing up, It's simply too dangerous, likely costs more than it earns, makes them look bad when it shows up, and alienates users who don't want to see it.

But those negative effects are probably all quantifiable to some dollar value that's less than the alternatives.

Executives don't get bonuses for altruism or ethical behavior. They get bonuses for making more money than they did last quarter. real fix is changing the game but that's a lot harder than it seems and can have many unintended consequences of it's own.

▲

happymellon 2 days ago | parent | next [-]

> 'd guess that the folks at Meta would absolutely love CSAM to stop showing up, It's simply too dangerous

Presumably not dangerous enough.

	▲	ticulatedspline 2 days ago \| parent [-]
		Which could be a "change to the game" but that might be worse. Protections like Section 230 are absolutely abused by large companies. But protections like that are also what has made the internet even possible Overly strict rules can also be weaponized. Imagine running a site that supports user content, but any appearance of CSAM on your site was jail-worthy, and you get an email demanding bitcoin or you're about to get hit with some posts that will shut you down. You have good moderation but you know you can't withstand a literal attack, someone who can automate that attack and has a financial motive to get illegal content onto your site. what do you do?

▲

FireBeyond a day ago | parent | prev [-]

You are talking about a company that willingly allowed one of its employees to be arrested when the government of that country was threatening to do so (apropos of the rightness or wrongness of that country in doing so), as a canary in a coal mine of sorts to decide whether they were willing to allow their executive team to travel there...

And then were okay with letting that person sit in jail longer because they thought it'd be better for Facebook PR if they did...

And then when they finally got that person out of jail, and later had him show up at a meet-n-greet with said executive team, Zuck didn't look up from his phone, looked confused, and said to his PA as he was leaving "who was that guy anyway, and why did he need to meet us?"

Or when Burmese dissidents, even the Burmese government itself, said that a FB feature would be used to track down and imprison, torture people the government didn't like, Zuck and FB leadership said "the feature is doing a lot to improve platform engagement, it needs to stay".

So the answer is worse. It's a combination of "the money" and pure psychopathy and sociopathy at the head of the dragon.

▲

Too 2 days ago | parent | prev | next [-]

> 36 billion ads needing review annually

This would mean every single user gets 10 uniquely personalized ads per year.

Even if that wasn't hilariously unreasonable by several orders of magnitude, if the moderation cost was was included in the cost of serving ads, this would all self regulate and only serious actors would participate. Treating it as a public bulletin board while still raking in a percentage of the impressions for sure is a nice business model, but not what's best for the rest of the society. Maybe ads just shouldn't be such a lucrative business.

▲

Cpoll a day ago | parent | prev [-]

> 550,000 moderators

You can bring this number down by only reviewing new/low-reputation advertisers, increasing minimum spend, rate limiting, etc.

You can also pre-filter for "definitely not CSAM" and only review the more ambiguous stuff.

I think your numbers are low too, surely a moderator can review over one ad per minute on average. And it may be more effective to split-screen multiple ads (with redundancy).

But at the end of the day it's a numbers game, something is going to slip through.