> Its conception of the world is only as good as the models and language (both mathematical and spoken) we have to express it.

Which is pretty damn good, all things considered.

And sure, training set text doesn't contain everything - but modern AIs aren't limited to just the training set text. Even in training stage, things like multimodal inputs and RLVR have joined the fray.

I don't think "create novel concepts" is a real limitation at all. Nothing prevents an AI from inventing new notations. GPT-4o would often do that when talking to AI psychosis victims.