I think it’s pretty clear that they are repeating without “comprehension” - both mechanistically (as in there is no facility for comprehension in their formulation) and in the ways they fail. The standard rs in strawberry, should I walk or drive to the car wash, etc all play on the fact that they don’t have any real world model or thoughts against which they can judge their output, as do many of the jailbreaks which basically play on the fact that the model has memorized patterns.

There are people who argue semantics, that we can call the pattern matching that LLMs do “understanding”, or the moronic “how do we know that’s isn’t all we do” but for the normal use of comprehension, LLMs at a fundamental level don’t.