Shouldn't that imply that agnosticism is the proper view, rather than asserting that something is impossible on a next-token-predictor architecture?

(Note: I don't actually think the consciousness question is the most important one in the near term. Where I think this line of reasoning gets really dangerous is when people use it to assert that LLMs can't or won't engage in certain behaviors no matter much they advance; this doesn't have anything to do with consciousness.)

I think you can reliably assert that X != Y without having a complete definition of Y, as long as you can identify at least one property or condition that Y possesses which X violates.

So for consciousness and LLMs it could be Qualia, lack of semantic understanding, lack of continuity in time, lack of a high degree of integrated causal feedback, etc.

Or perhaps those are just features of human consciousness but not integral to consciousness as a whole. To me this then implies panpsychism to some degree, which I'm alright with too.