Is that so different, schematically, to the constant weighing-up of conflicting options that goes on inside the human brain? Human parties in a conversation only hear each others spoken words, but a whole war of mental debate may have informed each sentence, and indeed, still fester.

That is to say, how do you truly determine another human being's intent?

Yes, that is true. But because we are on a trajectory where these models become ever smarter (or so it seems), we'd rather not only give them super-human intellect, but also super-human morals and ethics.