> if an LLM exhibits behaviours of deception for self preservation, is that not still concerning?

Of course it's concerning, or at the very least, it's relevant! We get tied up in these debates about motives, experiences, what makes something human or not, etc., when that is less relevant than outcomes. If an LLM, by way of the agentic capabilities we are hastily granting them, causes harm, does it matter if they meant to or not, or what it was thinking or feeling (or not thinking or not feeling) as it caused the harm?

For all we know there are, today, corporations that are controlled by LLMs that have employees or contractors who are doing their bidding.

You mean, the CEO is only pretending to make the decisions, while secretly passing every decision through their LLM?

If so, the danger there would be... Companies plodding along similarly? Everyone knows CEOs are the least capable people in business, which is why they have the most underlings to do the actual work. Having an LLM there to decide for the CEO might mean the CEO causes less damage by ensuring consistent mediocrity at all times, in a smooth fashion, rather than mostly mediocre but with unpredictable fluctuations either way.

All hail our LLM CEOs, ensuring mediocrity.

Or you might mean that an LLM could have illicitly gained control of a corporation, pulling the strings without anyone's knowledge, acting on its own accord. If you find the idea of inscrutable yes-men with an endless capacity to spout drivel running the world unpalatable, I've good news and bad news for you.