It solves for the same thing a "train of thought" (internal vocalization) solves for in the human brain: there's a lot of stuff you "know" as episodic memories of words you've heard/read, but haven't yet internalized via practice into reflex reactions. It's a lot faster and cheaper to acquire a memory of some words, than it is to acquire an entire new skill, so you end up picking up far more memories-of-words than concrete skills from a given training corpus.

Reasoning (i.e. bubbling those remembered words up into your mental context, and then using a special "trigger effectively arbitrary synaptic activations dependent on the content of the attention-masked context and an ongoing state" bridging reflex on them, a.k.a. "system-2 thinking") is how you leverage memories-of-words into useful behavior. Rather than executing native skills embedded into your synaptic weights, you're running a single very fancy native skill that acts as an interpreter for word-memory data, slowly and strenuously turning words you're holding in mind into more words (synthesis), or into associations to other memories (word-memories or otherwise), or into actions/outputs, etc.

(An especially useful "trick" of this system is planning, i.e. chaining the interpreter through "mere words" that conjure more "mere words" to mind until you end up at words that form a declaration of what you're about to do. You can then rely on a special part of the bridging reflex, namely the "instruction-following" reflex, to react in a way where your actions are constrained by the plan you've generated.)

▲

fedpost 4 hours ago | parent | next [-]

I get that this is the goal, but having looked at reasoning traces they kinda read like baby babble. The model is just kinda sitting there restating the question and itself over and over. It'll sometimes kinda state/clarify assumptions but it feels nothing like actual train of thought

▲

bitwize 5 minutes ago | parent | next [-]

It's copying a summary of what it needs to know to produce a set of tokens to solve the problem at hand to the front of its context window, so that it doesn't slide off the back and the model doesn't lose track of why it walked into the kitchen in the first place once it gets there.

▲

impossiblefork 2 hours ago | parent | prev | next [-]

That can make sense though.

I remember years ago now, trying to feed a text in twice etc., hoping that the representation in the KV cache of the second would end up better. I don't think it worked all that well.

But maybe that is most of the function of restating things babbling. To look at things multiple times.

▲

cowlevel 2 hours ago | parent | prev [-]

ISTR some researchers did a test replacing the reasoning tokens with ".", and reasoning still improved the results because dummy tokens are extra passes through the same network, allowing for more computation.

	▲	devmor 20 minutes ago \| parent [-]
		If you can find this research I would love to see it. That’s extremely interesting.

▲

bwestergard 4 hours ago | parent | prev | next [-]

Can you explain why your view, which is the opposite of the OP's, is correct?

"RL training directly raises the success probability , by spending a lot of compute on searching for successful paths and then training the model to reproduce them. The gains we credit to “thinking longer” come from the search we paid for during training, rather than at inference."

	▲	raincole 4 hours ago \| parent [-]
		Not mutually exclusive. LLMs are kind of like a self-fulfilling prophecy. Once some technique is discovered, it'd be part of the training process and becomes more efficient.

▲

dist-epoch 3 hours ago | parent | prev [-]

I think it also addresses the issue that there is a finite amount of computation you can do in one pass.

So you use one pass to compute one relevant word, let's say you are working on a sound analysis task, and you output "FFT".

Now in the second pass, FFT is right there at the input, so you can push that idea forward more until you are out of layers again and then you use the output token to prime the next pass.