I appreciate the counter argument, even as a chat bot skeptic.

However, there is the fact that you had to know how to poke the machine so it plucks the right vocabulary out of the training data.

There is very clearly no theory of mind: no inherent internalised modelling of how a human of a particular age thinks, speaks and what they do or do not know. These are the most obvious cracks in the “LLMs are (or will be) the superintelligence” narrative.

How hard is it to add that? We have plenty of writing from people of all ages to draw a model from. They just have not got round to it.