> people seem to develop very weird mental models of what LLMs are or do.

Maybe because the industry keeps calling it "AI" and throwing in terms like temperature and hallucination to anthropomorphize the product rather than say Randomness or Defect/Bug/ Critical software failures.

Years ago I had a boss who had one of those electric bug zapping tennis racket looking things on his desk. I had never seen one before, it was bright yellow and looked fun. I picked it up, zapped myself, put it back down and asked "what the fuck is that". He (my boss) promptly replied "it's an intelligence test". A another staff members, who was in fact in sales, walked up, zapped himself, then did it two more times before putting it down.

Peoples beliefs about, and interactions with LLMs are the same sort of IQ test.

> another staff members, who was in fact in sales, walked up, zapped himself, then did it two more times before putting it down.

It’s important to verify reproducibility.

Wow, your boss sounds like a class act