I think you may be interested in this discussion with Sam Harris and Cameron Berg.

https://www.youtube.com/watch?v=DRbZyuY8EN8

Berg is studying LLMs and working on more empirical frameworks for measuring LLMs capabilities regarding 'conscious-like' behaviors.

One of the more interesting things that was discussed is when you push an LLMs towards "You are just a machine with no consciousness" we start seeing higher rates of unaligned behaviors.

I'm really following these lines of research with interest. If they pan out then if LLMs are conscious or not is nearly irrelevant, if we want them to behave in alignment with human behaviors we'll have to train and treat them like they are. I don't really think this is any magical thing either, it's just framing the problem space correctly. For example, machines kill people all the time, it's not the machines fault, a human did something wrong. So telling an LLM it's just a machine may move it's output space from more moral like human decisions to more machine like amoral decisions.