But parent didn't try to apply "it's just maths" to humans. He said one could just as easily say, as some do: "Humans are just biology, hence they're not magic". Our understanding of mathematics, including the maths of transformer models is limited, just as our understanding of biology. Some behaviors of these models have taken researches by surprise, and future surprises are not at all excluded. We don't know exactly how far they will evolve.

As for applying the word thinking to AI systems, it's already in common usage and this won't change. We don't have any other candidate words, and this one is the closest existing word for referencing a computational process which, one must admit, is in many ways (but definitely not in all ways) analogous to human thought.