Some things that jump out as unfortunate:

- Reductionist analogies like how Microsoft Word is not conscious therefore AI is not.

- Dismissive in saying LLMs are not capable of moral reasoning. Maybe he meant agency or responsibility?

- Builds a case based on a Julius Caesar example without realizing its natural extension leads to a philosophical zombie which is not easy to disprove as consciousness.

- Seems to casually disregard non human sadness as a concern.

- Rolls out the stochastic parrot argument with a new coat of paint. This argument never made sense. Having a predictive element a part of the mechanism doesn’t rule anything out. Proper functionalist arguments tend to say what is required not what excludes.

Overall the article seems like a bit of a red herring. The premise is LLMs are not conscious, but most people don’t think they currently are so what’s the point of the claim?

The more interesting questions are whether artificial consciousness is fundamentally possible, and if so how far away are we and what pieces are still missing.