This highlights an interesting point that I hadn't examined before... the thoughts behind the words, and the experiences linked to those thoughts / the moment for which the words are needed.

Although from a chemical perspective and learning perspective humans may seem like stochastic machines, I think that the capability for inner thought and emotion does differentiate us from genAI. After all, we would not ascribe sentience to such a "stochastic parrot", but in a similar way we see the text output that AI generates and wrongly assume (in a previously reasonable assumption) that there must have been "thoughts" behind said text.

No different than a book. Maybe the places where the experiences were had were real and maybe they weren’t. Maybe we recall the emotion or it is projected from the story (or resonates in frequency with our own experiences).

I liken LLMs to https://en.wikipedia.org/wiki/Broca%27s_area

It's a functional subunit of the brain responsible for language interpretation and production.