language models are trained to maximise engagement and maximally convey meaning. binary contrast happens to achieve these goals by optimally translating the manifold. if I come across further research it would be interesting to continue our language related discussion.

>language models are trained to maximise engagement

I have seen no research to that effect.

>and maximally convey meaning

Nobody knows how to measure that

They are trained to complete sequences, then they are trained to engage in conversations, produce what people prefer (with a fairly crude measure of preference, hence the sycophancy). Reasoning models are trained to produce what an observer model think will give you the right answer, the observer model simultaneously learns the likelihood of producing the right answer.

All these things train to improve a property that can be measured at training time. You could build a model to estimate engagement, but I have not seen any evidence that shipping LLMs have used a measure like this.

I can't even imagine how you could tell if a measure of meaning was accurate or not, let alone how you could produce such a measure. You could try to measure information density, but that would also include noise.