I think the problem with this argument is that it's too inclusive. Is the bacteria that's adapted to an antibiotic conscious? It's showing intelligence right? I think if you're going to say something is potentially conscious, for me to take the argument seriously at least, there needs to be some plausible mechanism. I just don't see one for LLMs.

maybe the bacteria are conscious. How sure are we that they're not?

The only strong argument I have against it is the anthropic principle -- there are billions of times more bacteria than humans, so it's overwhelmingly unlikely that I'd be a human rather than a bacteria.

Not a very good argument of course.

Yeah I get what you are saying but I don't see much of a plausible mechanism for humans either and yet clearly there is one.