I agree with you that it’s a core question.

People don’t like it, however, for a whole host of reasons.

I don’t like it either, to be honest - for there the abyss may also stare into you - but I also often find that the things which we don’t like thinking about are critically important things to think about.

I also reach the same conclusion as you: it does not matter. If I cannot discern whether I am responding to a human- or LLM-written response, then we are back to zombie cats in boxes - and therefore the answer as to whether this precious magical spark we call consciousness (which may or may not exist anyway) exists in our interlocutor becomes moot.

And as I say - I may or may not be conscious. I seem to myself to be conscious, based on my understanding of the term - but I cannot prove that what goes on behind my eyes is the same as what goes on behind yours, or even that anything much is going on at all. Perhaps there’s just a narrative layer that likes to use “I” that parasitically explains the universe and the actions of the host to itself, and spreads between hosts through neurolinguistic programming and coadaptation. Maybe that’s what “we” are. I don’t know.

That said, perhaps I am wrong, and that is no small part for me of why this question should be earnestly considered and discussed. How can we possibly seek to understand or define machine intelligence before we examine our own.

For me consciousness simply means canvas-experience consciousness.

One of the things you learn in eastern style form of meditation is that "you" (the canvas-experience consciousness) are neither the source of your senses (obviously), nor of your ideas, nor of your emotions. All three are simply things that happen to "you". Even your ideas of "I" are generated and you experience them.

And yet the fact that the canvas-experience consciousness exists is the most irrefutable thing there is, because there IS experience. Is it a dream? Is it a simulation? Well, whatever it is, IT IS.

And just as that is obvious and irrefutable, it also seems pretty much impossible to show that anything other than yourself has this canvas-experience consciousness. As you say, you cannot know that behind my eyes there is this also.

Edit: Intelligence is also orthogonal to consciousness entirely. As I said your consciousness is not the generator of intelligent thoughts, so it is perfectly possible that intelligent thoughts can be generated by not-conscious systems, and for conscious systems to be not-intelligent.