>Neural networks were inspired by the brain, but transformers? It is totally plausible but do we really think just in words?

LLMs might be trained via words, but as a backend transformers are not just for words.

They're for high dimensional structured sequences. To make an analogy, transformers are not working on:

  Vector<Word>

  Vector<ContextualizedEmbedding>

And, we too, might not think in words, but I bet that we do think using multi-dimensional sequences/vectors.