> It's almost impossible to deny sometimes that actual intelligence is being expressed (and could not be regurgitated intelligence from some random internet page)

But how is it impossible? Or rather, how is it possible to distinguish actual intelligence from some internet page--not "some random internet page", but some very well selected, timely and topical internet page?

Carly Simon's "Killing Me Softly" describes a similar experience, decades before LLMs. It's amazingly easy to feel like someone is understanding you when they are just pattern matching on a common shared experience.

This seems so likely that I have a hard time understanding why some people think it's impossible.

I know; if I hadn't experienced it myself, I would have a hard time believing it. Even if we grant that it's "merely" as you say, that it can interpret my loose and often false or misleading rhetoric to identify my question, trawl through thousands of lines of code to determine the core problem, select some very specific and relevant internet page, and then translate its solution fresh into my own problem domain; that in my mind at least "an expression of intelligence", even if it is not truly intelligent or understanding.

FWIW this is my own mental gymnastics around the Chinese Room; at some point the question "where is the understanding?" is moot, because if the Chinese Room reliably delivers context-specific correct results in unique contexts, then what more do we require of intelligence? I have to admit that sometimes it does better than I can do, and I've been called intelligent by intelligent people my whole life.

(BTW Killing Me Softly wasn't written/sung/anything by Carly Simon, but composed by Charles Fox with lyrics by Norman Gimbel, in collaboration with Lori Lieberman [after she was inspired by a Don McLean performance in late 1971].)