I don't think the next-token-predictor thing should increase anyone's confidence that LLMs aren't conscious or can't escape the control of their operators. A very closely analogous argument would "prove" that humans aren't conscious or can't do [insert task here] either. (No, I'm not saying that any of this is true of today's LLMs, I'm saying this particular argument doesn't work.)

I recommend this explanation: https://www.astralcodexten.com/p/next-token-predictor-is-an-...

▲

dgellow an hour ago | parent [-]

You can say that for any argument regarding consciousness, because we don’t have an actual, all encompassing definition of what consciousness is. In general I don’t think comparison with humans makes much sense, we should be able to discuss LLMs without always falling back to “but what about humans” (sorry for the caricature)

▲

ameliaquining an hour ago | parent | next [-]

Shouldn't that imply that agnosticism is the proper view, rather than asserting that something is impossible on a next-token-predictor architecture?

(Note: I don't actually think the consciousness question is the most important one in the near term. Where I think this line of reasoning gets really dangerous is when people use it to assert that LLMs can't or won't engage in certain behaviors no matter much they advance; this doesn't have anything to do with consciousness.)

	▲	slopinthebag 34 minutes ago \| parent [-]
		I think you can reliably assert that X != Y without having a complete definition of Y, as long as you can identify at least one property or condition that Y possesses which X violates. So for consciousness and LLMs it could be Qualia, lack of semantic understanding, lack of continuity in time, lack of a high degree of integrated causal feedback, etc. Or perhaps those are just features of human consciousness but not integral to consciousness as a whole. To me this then implies panpsychism to some degree, which I'm alright with too.

▲

cyanydeez 23 minutes ago | parent | prev [-]

I think it's important to understand the humans _can_ do what LLMs do: predict next tokens from prior ones.

But LLMs are only operating on text and humans are only operating on <waves hands>