The first time someone fell for a phishing email that was LLM generated we had proof of AI telling convincing lies.

That was the very first thing that genAI was capable of.

If you’re ever bored, give an LLM a nonsensical word salad about “retrocausal computing algorithms” and have it write you up a whole thesis. Then think of something as far removed from the subject area as possible, and ask it to incorporate it in the analysis.

Without fail it will say something like “that is a natural next step” or “that’s an insightful connection” and go on to twist itself in contortions to draw connections between two completely nonsensical ideas.

If it’s capable of doing that, it is capable of lying, full stop.