David Deutsch on LLMs:

>LLMs are trained to imitate patterns of language, not to discover or verify truth. So, when asked to speak as an expert in an area where perceived experts have a widespread misconception, the LLM will parrot that misconception, adopting the register and vocabulary of experts.

>This is dangerous when people take that register and vocabulary as conferring authority, or the very nature of an LLM as conferring truth or impartiality, which it cannot. They then amplify and entrench the misconception.

Only distantly related to the subject, but it's in the general area of mental models. I suppose if you're trying to code a cliche, the LLM will help you get it right, and if you're trying to create something original, the LLM will help you get it wrong.