Some people here in the comments are arguing that the LLM "understands" what is "true" and "false", that is somewhat capable of reasoning, etc, but I still find it quite easy (with GPT-5) to break its facade of "reasoning".

I asked it to play a word game. This is very simple, and a very short session too. It failed in its very first response, and then it failed in explaining why it failed. All with total confidence, no hesitation.

Nobody fluent in English would fail so catastrophically. I actually expected it to succeed:

https://chatgpt.com/share/68bcb490-a5b4-8013-b2be-35d27962ad...

It's clear by this failure model the LLM doesn't understand anything.

Edit: to be clear, as the session goes longer it becomes more interesting, but you can still trip the LLM up in ways no human "understanding" the game would. My 6-year old plays this game better, because she truly understands... she can trip up, but not like this.