Isn't the whole point of attention, some context comprehension in the stochastic parrot machine.

Yes, however I do suspect that at some point comprehension trumps context. Meaning it will be evedent giving the ability to increase comprehension and retaining whatever context you have goes a long way. I dont have the ability to play such large models. Within the means of my hardware I have already playd around with LLM enough to know where balances will start to cause confliction. Priotizing effeciency within the accuracy. This can mean compromising some other aspect of the AI, or this can be viewed as a positive interoperability.