mdp2021 7 days ago

But a big point here becomes whether the generated "context" then receives proper processing.

What processing? When you have an internal line of thought, what processing do you do on it?

For me, it feels like I say something, and in saying it, and putting it into words, I have a feeling about whether it is true and supported or not. A qualitative gauge of its correctness. A lot of my reasoning is done this way, trusting that these feelings are based off of a lifetime experience of accumulated facts and the another things currently being considered.

Explain to me how this is different than a neural net outputting a weight for the truthiness of the state space vector?

	▲	mdp2021 6 days ago \| parent [-]
		I meant the LLM. I meant, with «whether the generated "context" then receives proper processing», whether the CoT generated by the LLM and here framed as further "context", regardless here of how properly it is generated, receives adequate processing by the internals of the NN. A good context (any good context) does not necessarily lead to a good output in LLMs. (It does necessarily lead to a better output, but not necessarily a satisfying, proper, decent, consequential one.)