It's not that simple.

Functionally speaking, i.e. talking about observable output and not making any claims about LLM "mental states", there are plenty of examples where LLMs demonstrate good theories of the world and theory of mind. They can even exhibit, again from a functional perspective, emotional intelligence.

However, they need input to tell them what the world in question is like and what the minds are like. Otherwise, they're just operating from an enormous amount of often-conflicting pretrained data.

You're more likely to see good results in this area when interacting directly with one, so that it has some history of the interlocutor's writing, or when doing something like coding where "the world" is something like a codebase, along with the pretrained information it has about that kind of code.

Code is a good example, because the results they give in that context would not be possible without some equivalent of a theory of the subject. This applies to pretty much any kind of reasoning that models do - in math, logic, physics etc.

When writing for a more general audience, models lack this kind of information, so you would have to supply it in some form.

The other issue is that whatever theories are implied by their writing are not necessary stable, because of issues like context size limitations, compaction, selective attention, etc. Nevertheless, it's a big oversimplification to say that they don't possess the functional equivalent of such theories.