I get what you are pointing at: you are focusing with some strictness on the post from Stavros, which states that "people pseudo-rationalize with plausible explanatory theories their not-at-the-time-rational behaviour".

But I was instead focusing at the general problem in the root post from Foundry27, and to a loose interpretation of the post from Stavros: the opposition between the faculty of generating convincing fantasies vs the faculty of critical thinking. (Such focus being there because more general and pressing in current AI than the contextual problem of "explanation", which is sort of a "perversion" when compared to the same in classical AI, where the steps are recorded procedurally owing to transparency, instead of the paradox of asking an obscure unreliable engine "what it did".)

What I meant is that a general scheme of bullshitting to oneself and pseudo-rationalizing it is not the only way. Please see the other sub-branch in which we talked about mathematics. In important cases, the fantasies are then consciously checked as thoroughly as constraints allow.

So I stated «/Some/ people bullshit themselves stating the plausible; others check their hypotheses ... Some people will output a pre-given answer; some people check» - as a crucial discriminator in the natural and artificial. Please note that the trend in the past two years has generated a believe in some that the at most preliminary part is all that there is.

Also note that Katskul wrote «only occasionally do we deviate to check ourselves» - so the reply is "No: the more one is educated and intellectually trained, the more one's thoughts are vetted - the thought process is disciplined to check its objects".

But I see re-checking the branch that the post from Stavros was strongly specific towards the "smaller" area of "pseudo-rationalizing", so I see why my posts may have looked odd-fitting.

On my having come with percussions at the strings meeting see the other reply.

I want to check the papers you proposed as soon as I will have the time: I find it difficult to believe that the conscious cannot intercept those "changes of mind" and correct them.

But please note: you are writing «Not only are ... not intentional»... Immature thought needs not to be intentional at all: it is largely spontaneous thought. But whether part of an intentional process ("let us ponder towards some goal"), or whether part of the subterranean functions, when it becomes visible (or «intercepted» as I wrote above), the trained mind looks at it with diffidence and asks questions about its foundations - intentionally, in the conscious, as a learnt process.

I have no illusions on LLMs, I have been working with them since og BERT, always with these same issues and more. I'm just stating what would be needed in the future to make them reliably useful outside of creative writing & (human-guided & checked) search.

If an LLM provides an incorrect/orthogonal rhetoric without a way to reliably fix/debug it it's just not as useful as it theoretically could be given the data contained in the parameters.