> While a human may say “aha” to indicate exactly a sudden internal state change, this interpretation is unwarranted for models which do not have any such internal state, and which on the next forward pass will only differ from the pre-aha pass by the inclusion of that single token in their context. Interpreting the “aha” moment as meaningful exemplifies the long-neglected assumption about long CoT models – the false idea that derivational traces are semantically meaningful, either in resemblance to algorithm traces or to human reasoning.

This paper addresses something that has always bothered me about LLMs. You read their reasoning, see something like “Wait, that’s wrong” and then watch them make the exact mistake they just identified.

▲

Jeff_Brown a day ago | parent [-]

By itself, "aha" carries no insight, but the insight is probably stated immediately after it. In that case the aha is semantically useful, by identifying the insight it is near.

▲

deaton 2 hours ago | parent | next [-]

It really isn't useful though, unless it is a summary. At best it is a semantic trick to tell the next iteration to come up with something smart.

▲

paimapi a day ago | parent | prev | next [-]

it's a rhetorical heuristic that a writer should know to use when directing a reader to a declarative that they want them to pay attention to, usually because it's a non-obvious or roundabout insight

when utilized by AI, it's a probabilistic output and it's variable whether or not that rhetorical trick is useful. it also pushes a non-skeptical reader to focus too much on the following text or even to believe that they, themselves, derived some insight. this is effectively a kind of persuasive sophistry which is not helpful - adding rules around it prevents people from deluding themselves with AI

	▲	ghostpepper 2 hours ago \| parent \| next [-]
		Did not read the paper so apologies if this is covered but isn't it possible that there is some recognizable semantic pattern in the training data where an "aha" is often followed by a subtle semantic shift that proves closer to the original premise in some critical way, and by emitting the "aha" token the model causes itself to produce such a subtle semantic shift that pushes the subsequent reasoning closer to the desired response?
	▲	abitmoa 19 hours ago \| parent \| prev [-]
		It amounts to noise overall, but it has further unwanted and potentially misleading 'properties'. I think it's rather sobering to see how much bandwidth is still being wasted.

▲

wizzwizz4 21 hours ago | parent | prev [-]

> but the insight is probably stated immediately after it.

If the intermediate tokens represent reasoning or thought, you would expect "aha" to occur after the thoughts that led to the realisation, including the thoughts encoding the explanation: they don't have any other state. There is no reason to draw the conclusion you've drawn. Furthermore, what LLMs are doing isn't thought.