Is the above comment a genuine question? I’m concerned it’s a rhetorical question that isn’t really getting to the heart of the matter; namely, what is the empirical performance? One’s ability to explain said performance doesn’t always keep up.

How about we pick an LLM evaluation and get specific? They have strengths and weaknesses. Some do outperform humans in certain areas.

Often I see people latching on to some reason that “proves” to them “LLMs cannot do X”. Stop and think about how powerful such a claim has to be. Such claims are masquerading as impossibility proofs.

Cognitive dissonance is a powerful force. Hold your claims lightly.

There are often misunderstandings here on HN about the kinds of things transformer based models can learn. Many people use the phrase “stochastic parrots” derisively; most of the time I think these folks are getting it badly wrong. A careful reading of the original paper is essential, not to mention follow up work.

I'm not making a blanket statement against LLMs for all use cases. I'm certain that LLMs are, for example, much more performant at indexing already-curated documents and locating information within them than humans operating manually are.

What I'm skeptical about isn't LLMs as a utilitarian tool to enhance productivity in specific use cases, but rather treating LLMs as sources of information in their own right, especially given their defining characteristic of generating novel text through stochastic inference.

I'm 100% behind RAG powering the search engines of the future. Using LLMs to find reliable sources within the vast ocean of dubious information on the modern internet? Perfect -- ChatGPT, find me those detailed blog posts by people competent in the problem domain. Asking LLMs to come up with their own answers to questions? No thanks. That's just an even worse version of "ask a random person to make up an answer on the spot".