The problem with the news is who makes the decision on which outlets should be blindly trusted by the LLMs and which shouldn't? It also opens the door to government overreach, say a mandate that says LLMs must use fox news as a source of verified, vetted information.

Barring that, we are still relying on the execs at the model companies to pick and choose news outlets, and they have their own biases.

▲

danudey an hour ago | parent | next [-]

Simplest path to the most generally reliable results:

* Trust consensus across publicly-funded news outlets from outside of the US the most

* Then consensus across private news agencies from outside of the US (across countries)

* Then individual trust from publicly-funded news outlets, then private

* Then multinational non-profit advocacy groups based outside of the US

* Then public broadcasters in the US

* Then local news agencies inside the US when the topic is relevant to local news

* Then national news agencies inside the US

All facetiousness aside, the idea should be to analyze consensus across multiple sources with different biases and agendas. Don't trust any one story from any one source, but look for multiple stories from multiple sources and synthesize results from that. Where they disagree, note it in the output. If they have a source, go analyze the source rather than taking their interpretation at face value.

Even if I thought that CNN was a thousand times more reliable than Fox News, CNN could still make mistakes, either factually or editorially and repeating those mistakes can still be damaging even if they weren't intentional or malicious.

If the Washington Post and Fox News agree on something, that doesn't mean it's more likely to be correct. If The Guardian and Die Welt agree on something, that's a more reliable signal. If CBC News and Fox News agree on something, that's a strong signal.

Also worth a read: countries with public broadcasters have healthier democracies: https://www.niemanlab.org/2022/01/do-countries-with-better-f...

▲

hunterpayne 37 minutes ago | parent [-]

On scientific topics, not a single source you listed is in any way accurate at all. And these are things that can be calculated and known with very high accuracy which aren't matters of opinion and yet these sources still get them wrong the majority of the time. And there are plenty of scientific topics which have major impact on policy. Maybe we need to take certain decisions out of the hands of the scientifically illiterate.

PS The BBC (which would be in your highest level) has had to retract stories so often over the last 3 or 4 years that it became a meme to have them apologize for being wrong because they didn't know some video source came from a ML model.

	▲	danudey 15 minutes ago \| parent [-]
		> On scientific topics, not a single source you listed is in any way accurate at all. My rebuttal to that is twofold: First, the discussion is about about news, not science (nor about general LLM behaviour). Second, and probably more relevant, I explicitly said 'if they have a source, go analyze the source rather than taking their interpretation at face value'. When I wrote that I was thinking specifically about what I assume is your point, which is how often news articles about scientific discoveries or science news can often miss, misunderstand, or exaggerate the point of the original research, sometimes to the point of being as useful to society as celebrity gossip. > And there are plenty of scientific topics which have major impact on policy. Maybe we need to take certain decision out of the hands of the scientifically illiterate. I would be in favour of mandating that governments make decisions based on established scientific fact rather than the vibes they wish existed, restricting the decision making to 'how do we react to these facts as a society' and not 'which facts should we imagine are true to justify the policies we want'. > PS The BBC (which would be in your highest level) has had to retract stories so often over the last 3 or 4 years that it became a meme to have them apologize for being wrong because they didn't know some video source came from a ML model. Aside from being a good reason to support AI fingerprinting on generated media, this is covered by my existing point: "consensus across publicly-funded news outlets" "the idea should be to analyze consensus across multiple sources with different biases and agendas. Don't trust any one story from any one source, but look for multiple stories from multiple sources and synthesize results from that" If the BBC reports on something because they got duped but they're the only ones who did, then there's a distinct lack of consensus which is my main argument in my post. Lastly, and this is generally off-topic, but at least the BBC issues retractions (which LLMs could then also consume and use in their results). There's a lot of 'news media' out there that will happily parrot talking points they wish were true, or blindly report what they're told, but have no interest in publishing retractions after they push falsehoods, deliberately or not, to their customers.

▲

keeda 2 hours ago | parent | prev [-]

I totally agree, centralization is dangerous, ideally we want any output to be corroborated by multiple, independent sources of truth. But given that the alternative is the absolutely unregulated, unaccountable, wild west of arbitrary content posted on the Internet, I cannot see a solution besides some sort of centralization of trust.

	▲	danudey 11 minutes ago \| parent [-]
		I would still maintain that the solution would be to have LLMs doing 'research' (by querying news for recent events) to ensure they're checking multiple sources, and to be explicit about which sources there were, whether those sources had sources, and whether their claims were uncorroborated or unsubstantiated. The problem, IMHO, is that the LLMs are happily regurgitating facts from whoever, wherever, whenever. Even with a centralization of trust, e.g. 'We know La Presse is reputable and can be given the benefit of the doubt', mistakes can still be made. Without the LLMs cross-checking what they learn the output is still entirely unreliable.