I'm not making a blanket statement against LLMs for all use cases. I'm certain that LLMs are, for example, much more performant at indexing already-curated documents and locating information within them than humans operating manually are.

What I'm skeptical about isn't LLMs as a utilitarian tool to enhance productivity in specific use cases, but rather treating LLMs as sources of information in their own right, especially given their defining characteristic of generating novel text through stochastic inference.

I'm 100% behind RAG powering the search engines of the future. Using LLMs to find reliable sources within the vast ocean of dubious information on the modern internet? Perfect -- ChatGPT, find me those detailed blog posts by people competent in the problem domain. Asking LLMs to come up with their own answers to questions? No thanks. That's just an even worse version of "ask a random person to make up an answer on the spot".