> But it is another good example that "AI" is just glorified search and there is not reasoning or thinking going on behind the covers.

I don't think that follows. This is just LLMs being, for a lack of a better word, "gullible." How is it different from a person believing whatever they read on the Internet? People fall for spam and scams all the time, doesn't mean they are just glorified searches ;-)

It does highlight the problem facing any search engine though. AI-generated spam will be much harder to defend against with traditional, statistical mechanisms. And this is before we get to the existential problem of prompt injection.

Maybe this is where news organizations can win back their proper place in their relationship with Big Tech: by becoming the sources of verified, vetted information that LLMs can trust blindly. Possibly that's what deals like the OpenAI / Atlantic one are about.

> How is it different from a person believing whatever they read on the Internet?

The problem is LLMs have no capacity for shame.

My Dad got taken in by a Target gift card scam. He felt so terrible, he almost didn't even tell me about it. He may get scammed again, but not by anything remotely like that.

To LLMs, all mistakes just get washed together into the same bucket. They don't spend days feeling depressed and stupid over getting scammed. There's no giant blinking red light that says, "Never let this happen again!"

The problem with the news is who makes the decision on which outlets should be blindly trusted by the LLMs and which shouldn't? It also opens the door to government overreach, say a mandate that says LLMs must use fox news as a source of verified, vetted information.

Barring that, we are still relying on the execs at the model companies to pick and choose news outlets, and they have their own biases.

"How is it different from a person believing whatever they read on the Internet?"

Because a person is alive while the LLM is a floating point number database with a questionable degree of determinism.

>"gullible."

Enough with the anthropomorphization