The disanalogy you quote might actually be the key insight. What if language operates at two levels, like Kahneman's System 1/2?

Level 1: Nearly autonomic — pattern-matched language that acts directly on the nervous system. Evidence: how insults land before you "process" them, how fluent speakers produce speech faster than conscious deliberation allows, and the entire body of work on hypnotic suggestion, which relies on language bypassing conscious evaluation entirely.

Level 2: The conscious formulation you describe — the translator between perception and meaning.

LLMs might be decent models of Level 1 but have nothing corresponding to Level 2. Fedorenko's "glorified parser" could be the Level 1 system.

▲

lapcat 4 days ago | parent [-]

> LLMs might be decent models of Level 1

I don't think so. Fast speakers and hyponotized people are still clearly conscious and "at home" inside, vastly more "human" than any LLM. Deliberation and evaluation imply thinking before you speak but do not imply that you can't otherwise think while you speak.

▲

adamzwasserman 4 days ago | parent [-]

The body of knowledge on Ericksonian hypnotherapy is pretty clear that the effect of language on Level 1 is orthogonal to, and sometimes even opposed to, conscious processes.

I became interested after being medically hypnotized for kidney stone pain. As the hypnotist spoke, I was consciously thinking: "this is dumb, it will never work." And yet it did.

That's exactly your point — I was fully conscious and "at home" the whole time, yet something was processing and acting on the language independently. The question is whether that something shares any computational properties with LLMs, not whether the whole system does.

▲

throaway123213 4 days ago | parent | next [-]

"It's exactly your point — I was fully conscious and "at home" the whole time, yet something was processing and acting on the language independently."

It's unclear what you're referring to here. You were conscious, & you wanted to think the thought "this is dumb, it will never work." & you thought that. What was the independent process?

	▲	adamzwasserman 4 days ago \| parent [-]
		The hypnotism worked. There was an unconscious process at work whose relationship to the words was completely different from my conscious reaction.

▲

lapcat 4 days ago | parent | prev [-]

I think you're creating a false dichotomy between meta-thinking and mere reflex, when in fact most conscious thinking is neither of those.

My understanding is that a hypnotized person is very focused on the hypnotist and suggestible but can otherwise carry on a relatively normal conversation with the hypnotist. And certainly an unhypnotized chattering person is still conscious, aware of the context as well as the subject of their speech. You may find the speech dull and tedious, may even call it "mindless" as insult, yet it's honestly impossible to dispute that there's an active human mind at work.

▲

adamzwasserman 4 days ago | parent [-]

I don't think we're far apart. My claim isn't that Level 1 is "mere reflex". It's that language can produce effects at a level that operates independently of (and sometimes in opposition to) conscious evaluation. The hypnosis example is just a clean demonstration of that separation.

Whether LLMs are useful models for studying that level is an empirical question. They're not conscious, but they do learn statistical regularities in language structure, which may be exactly what Level 1 is optimized for.

	▲	lapcat 4 days ago \| parent [-]
		> language can produce effects at a level that operates independently of (and sometimes in opposition to) conscious evaluation I don't think this is a particularly interesting claim if "conscious evaluation" is understood so strictly that it excludes an ordinary motormouth.