Then I think I'd rephrase you: There's a reason we trust trusted systems.

:)

Even their fans agree, LLMs are inherently unreliable. The fact some people trust them regardless is due to a deep flaw in human psychology that I believe has not been significantly exploited by any previous tech. There will be tears.

I think two things contribute to the trust in LLMs:

1. Decades of generally reliable systems (I trust my calculator because it always says 1+1=2) has trained people to believe what computers say.

2. LLMs "speak" with great confidence, which has, as you say, a psychological effect.