If you loop an entire transformer model on itself, that seems like by-definition hidden reasoning.

If the output of the model is its reasoning trace, and you simply feed that back into the model again at inference time instead of outputting it - then it is by definition hidden (but I would expect you could pull both this trace and a further-down final output trace out)

▲

WhitneyLand 6 hours ago | parent | next [-]

No. It’s not at all by definition hidden reasoning.

Looping transformers uses additional calculations (repeating layers) to generate a token.

Reasoning (in this context) is test time generation of multiple tokens that allow a model to have a scratch pad to refine its thoughts, chain of thought reasoning in other words.

Doing the former in no way means that you have to hide the latter.

Raschka is right in this post, The Information article was wrong. The Astra system card does concede reasoning traces are sometimes smaller, but this could be for a lot of reasons, including simple efficiency. And it absolutely doesn’t mean they are going away or completely obscured.

The Last Week in AI podcast from Sept 8 seems to have gotten this wrong as well. Jeremie Harris rages that OpenAI implemented latent reasoning, ala the coconut paper, which could potentially actually obscure reasoning traces. But for the life of me, I do not know how he arrived at this conclusion and see no evidence that this has happened in Astra.

	▲	SubiculumCode 3 hours ago \| parent \| next [-]
		Interesting. I admit I was largely going on what Mr. Harris said on the episode when responding to several posts.
	▲	famouswaffles 5 hours ago \| parent \| prev [-]
		It can lead to hidden reasoning, if the looping allows it to stuff enough information outside visible CoT. Open AI demostrates such an ability by asking it to solve problems while thinking about something else entirely. All the other models are unable to do this except Astra. It doesn't have to be a substitute for CoT to cause monitorability issues.

▲

XenophileJKO 6 hours ago | parent | prev [-]

Not really, only if you pass a residual down to the next time step.

There isn't really anything fundamentally different compared to a similar depth traditional "unrolled" model. It helps with parameter efficiency.

That doesn't mean that the model can't have "hidden" internal state, it just means it has to recompute the "hidden" part on every token inference pass without outputting it, or learn a subversive alternate meaning to words in the thought space.

This is why you see openai say that they don't want to apply direct optimization pressure on thought traces because the more the you penalize "bad thoughts" the more it could put maladaptive pressure on the reasoning tokens where they may learn "subversive meanings". It effectively damages monitoring.

Like thinking "look at" when you really mean "hack into" or even more radical coded language.