> you build a world model

The foundational conceit (if you will) of LLMs is that they build a semantic (world) model to 'make sense' of their training. However it is much more likely that they are simply building a syntactic model in response to the training. As far as I know there is no evidence of a semantic model emerging.

Maybe I don’t have a precise enough definition of syntax and semantics, but it seems like it’s more than just syntactic since interchangeable tokens in the same syntax affect the semantics of the sentence. Or do you view completing a prompt such as “The president of the United States is?” as a syntax question?

There's some evidence of valid relationships: you can build a map of Manhattan by asking about directions from each street corner and plotting the relations.

This is still entirely referential, but in a way that a human would see some relation to the actual thing, albeit in a somewhat weird and alien way.

Is this not addressed by othellogpt?