I like the idea of more reccurance in the transformer level. Chain of thought always seemed so clunky. Its just not the way the human brain processes information. Its an extrmeely crude approximation at best

▲

sznio 7 hours ago | parent | next [-]

it is what I do to solve hard problems through.

easy stuff happens by itself, but with a system large enough you need a scratchpad and a rubber duck.

	▲	dgellow 7 hours ago \| parent \| next [-]
		One thing about the reasoning is that models are trained to generate a chain of thoughts, but it doesn’t have to be correct, accurate, or reflect the underlying logic of the LLM. It’s the same problem we have with the output, it is something plausible, but not that reliable
	▲	naasking 6 hours ago \| parent \| prev [-]
		Yes, both the output should be "milestones" of sorts, like lemmas and theorems in math. Important plateaus that serve as a launching pad to the next phase. Regurgitating every thought potentially degrades signal:noise ratio.

▲

big_toast 5 hours ago | parent | prev [-]

Chain of thought seems very specifically like an instantiation of the System 2 thinking of the System 1 and 2 thinking Daniel Kahneman popularized.

Is it clunky in that it's a verbalized/languified version of system 2 thinking, and clearly humans do some non-verbal version too?

	▲	ElFitz 5 hours ago \| parent [-]
		Some humans apparently only or almost only do non-verbal thinking. It’s called anendophasia.