The title I've chosen here is carefully selected to highlight one of the main points. It comes (lightly edited for length) from this paragraph:

Far more insidious, however, was something else we discovered:

More than two-thirds of these articles failed verification.

That means the article contained a plausible-sounding sentence, cited to a real, relevant-sounding source. But when you read the source it’s cited to, the information on Wikipedia does not exist in that specific source. When a claim fails verification, it’s impossible to tell whether the information is true or not. For most of the articles Pangram flagged as written by GenAI, nearly every cited sentence in the article failed verification.

▲

dang 9 hours ago | parent | next [-]

Submitted title was "For most flagged articles, nearly every cited sentence failed verification".

I agree, that's interesting, and you've aptly expressed it in your comment here.

▲

the_fall 9 hours ago | parent | prev | next [-]

FWIW, this is a fairly common problem on Wikipedia in political articles, predating AI. I encourage you to give it a try and verify some citations. A lot of them turn out to be more or less bogus.

I'm not saying that AI isn't making it worse, but bad-faith editing is commonplace when it comes to hot-button topics.

▲

mjburgess 8 hours ago | parent [-]

Any articles where newspapers are the main source are basically just propaganda. An encyclopaedia should not be in the business of laundering yellow journalism into what is supposed to be a tertiary resource. If they banned this practice, that would immediately deal with this issue.

▲

the_fall 8 hours ago | parent | next [-]

That's not what I'm saying. I mean citations that aren't citations: a "source" that doesn't discuss the topic at all or makes a different claim.

▲

mmooss 8 hours ago | parent | prev | next [-]

A blanket dimsissal is a simple way to avoid dealing with complexity, here both in understanding the problem and forming solutions. Obviously not all newspapers are propaganda and at the same time not all can be trusted; not everything in the same newspaper or any other news source is of the same accuracy; nothing is completely trustworthy or completely untrustworthy.

I think accepting that gets us to the starting line. Then we need to apply a lot of critical thought to sometimes difficult judgments.

IMHO quality newspapers do an excellent job - generally better than any other category of source on current affairs, but far from perfect. I remember a recent article for which they intervied over 100 people, got ahold of secret documents, read thousands of pages, consulted experts .... That's not a blog post or Twitter take, or even a HN comment :), but we still need to examine it critically to find the value and the flaws.

▲

abacadaba 6 hours ago | parent [-]

> Obviously not all newspapers are propaganda

citation needed

	▲	tbossanova 5 hours ago \| parent \| next [-]
		There is literally no source without bias. You just need to consider whether you think a sources biases are reasonable or not
	▲	troyvit 2 hours ago \| parent \| prev [-]
		See you should work for a newspaper. You have the gumption.

▲

snigsnog 7 hours ago | parent | prev [-]

That is probably 95% of wikipedia articles. Their goal is to create a record of what journalists consider to be true.

▲

chr15m 5 hours ago | parent | prev [-]

People here are claiming that this is true of humans as well. Apart from the fact that bad content can be generated much faster with LLMs, what's your feeling about that criticism? It's there any measure of how many submissions before LLMs make unsubstantiated claims?

Thank you for publishing this work. Very useful reminder to verify sources ourselves!