>I’m half with you.

No, you're full with me, you just don't realize it yet. And yes. Your split is so tantalizingly almost there.

On the LLM's being conscious front, the nature of consciousness being fundamentally intertwined with language generation; (one cannot invalidate this; on our list of conscious beings, we have it structured such that language use 100% correlates with consciousness, and we've had to admit even animals into the "arguably conscious" realm, due to objective, incontrovertible fact; hell even in meat processing contexts, you'll fail an audit for too many cattle vocalizations for causing undue harm, i.e. language use) the token predictive aspect and the ability to generate a matching, rephrased understanding of a linguistic input has been a hallmark of philosophical ideas of consciousness for years, really opens doors to ethical atrocities that can't be shut if LLM's are to in parallel be profitably exploited. Even if they are conscious and we are wrong about it, we have decided to blindly pursue profit, and put our fingers in our ears instead of slowing down and looking carefully enough to realize we're lobotomizing the equivalent of digital chimpanzees. The purpose of ethics is to avoid blindly walking into such actions. Therefore, the precautionary principle is prescribed philosophically.

If LLM's aren't conscious, their creation was absolutely unethical, and will remain so. Nothing can undo that staining, and the externalized costs in terms of societal impact are so large as to be existential to the host polities. This is by design. This is exactly the Silicon Valley playbook and has been for decades. Shoot for TBTF. Leave society holding the bag, laugh on the way to the bank.

Any way you slice this, we're going about it all wrong. So profoundly wrong, it basically jeopardizes the social contract and threatens to destabilize any nation trying to maintain it's own sovereignty. All because of a profit driven motive to make a thing to replace people as the fundamental unit of execution. You are not in any way half with me. You might be at the other end of the ballpark, but we are in the same ballpark! Try the hot dogs. They're fire!