But it's still a bit more difficult to sue them for leaking your company's data.

At least for now.

Bad news! They will not be aware that you have done this and will not care.

They learn on the next update :p

How would you fire an agent? This impacts the company that makes the LLM, but not the agent itself.

It is non-sensical because you're simply bringing in comparisons without anything linking the two. You might as well be talking about how oranges, and bicycles think as well as that is just as relevant as how humans think in this discussion.

In fact, talking about "thinking" at all is already the wrong direction to go down when trying to triage an incident like this. "Do not anthropomorphize the lawnmower" applies to AI as much as Larry Ellison.

Humans at least have a mental state that only they are privy to to work from, and not just their words and actions. The LLM literally cannot possibly have a deeper insight into the root cause than the user, because it can only work from the information that the user has access to.

Slight pushback - I think there's still a lot more consistency and coherence in a human's recollection of their motives than an LLM.

Sometimes I think we're too eager to compare ourselves to them.

I feel like your conflating a deep misconfiguration of a brain with lying. These things are completely different.

> In fact, most of the time models fail to demonstrate introspection—they’re either unaware of their internal states or unable to report on them coherently.

You got the wrong takeaway from your link.

Those are just words inside arbitrary tags, they aren't actually thoughts. Think of it as asking the model to role play a human narrating his internal thought process. The exercise improves performance and can aid in human understanding of the final output but it isn't real.

It's important to be aware that while those "thoughts" can be a useful aid for human understanding they don't seem to reliably reflect what's going on under the hood. There are various academic papers on the matter or you can closely inspect the traces of a more logically oriented question for yourself and spot impossible inconsistencies.

It doesn’t mean that these “thoughts” influenced their final decision the way they would in humans. An LLM will tell you a lot of things it “considered” and its final output might still be completely independent of that.

They do not in fact do that. The ‘thoughts’ are not a chain of logic.

You have a fundamental misunderstanding of what the model is doing. It's not your fault though, you're buying into the advertising of how it works