A probably unacceptably large portion of the population DOES think they’re infallible, or at least close to it.

Totally. I get screenshots from my 79yo mother now that are the Gemini response to her search query.

Whatever that says is hard fact as she's concerned. And she's no dummy -- she just has no clue how these things work. Oh, and Google told her so.

That may be true, but the underlying problem is not that the LLMs are capable of accurately reporting information that is published in a single person's blog article. The underlying problem is that a portion of the population believes they are infallible.