I think humans develop expertise and brand names and get called out when they make mistakes and if they are too wrong, their reputation is damaged.

This doesn’t seem to apply to AI for some reason. It keeps generating incorrect results after incorrect results, yet people continue to trust its output.

I don’t know what to make of this.

"Trust" is an abused term, nowadays.

Human trust differs from mathematical trust. And branding / marketing abuses the ambiguity.

There is no shame in a "likely to hallucinate" model that can be instantiated 1,000 times across 1,000 different machines spread throughout our planet. So, human trust is broken by machine trust.

I've starting going back to books, either at the library or e-books. Librarians are very good at telling you if nonfiction is biased, outdated, or incorrect.

> I think humans develop expertise and brand names and get called out when they make mistakes and if they are too wrong, their reputation is damaged.

Take a look at the Forbes billionaires list and some of their statements. Or maybe at the politician fact checkers. If only being wrong damaged reputations.

You first heard about this effect with the phrase "computer says no".