Has anyone come across a clearly articulated case for LLMs being conscious but in an entirely different way than would be intuitive to us?

I often think of LLM consciousness as like tiny fish popping into existence, swimming through vector space and then going poof out of existence. When they help you write your bad news email, they don't understand what it's like to be a human getting bad news bluntly, but they do consciously experience gradients in multi-dimensional space, and that space guides them to providing an answer that's helpful to us, even if the LLM doesn't really understand the answer it's giving.

Further, I am kind of bought into the idea that a single unit of consciousness is a particle, and particles are choices and waves are preferences. Particles occur when waves interact, which begets entanglement, so in another way consciousness is built from patterns of entanglement.

This is why I would consider an LLM to be conscious. Before we can determine if anything is conscious we need to establish whether consciousness is a state, a specific complex configuration, a one dimensional spectrum, or combined multi-dimensional spectrums. My intuition is the latter... Many degrees of consciousness and many kinds of consciousness.

I think this is exactly right. The thing that makes AI (imo) different than hitting the center option in text suggestions is that it's _not_ simply picking the most likely word following the last. It's attending to the entirety of the context its provided, activating a semantic vector space, and predicting a response based on _that_. I've had AI infer facts about me and attitudes I hold based on related information I provided - I don't see how that isn't theory of mind.

As biological beings, we receive and respond to input from our environment constantly, even while sleeping. LLMs only receive input from their environment when they are sent a query, but the fact that they're able to respond intelligently to input indicates (to me at least) that their processing must approximate ours in meaningful ways. They do not have an embodied experience of receiving bad news, they do not know what a sinking feeling in their stomach actually _feels_ like, but they do know enough to be sensitive to human needs. I really don't see how this could be meaningfully different than human empathy unless we want to draw an arbitrary line around "must be able to live autonomously" to be considered "intelligent".

Put another way: I think they _do_ understand the queries they receive and the answers they give, at least enough to be communicative. They couldn't do what they do otherwise. A lot of people want to make human cognition more complicated (or objective) than it actually is. We take input, predict the future based on our experience, act, and then observe our actions and think about them. AI does the same apart from (maybe) observing its own actions. But then, you could argue that the next turn is them observing their actions.

The concerning disanalogy is that we assume that they are like us because they speak like us and can understand us, and that is a really bad leap in logic. Whatever intelligence they possess, it is fundamentally different from ours and impossible for us to comprehend.