The CoT shows Astra skipping spaces and words. I have never seen this before. Is it a consequence of the looped architecture?

▲

welferkj 5 hours ago | parent | next [-]

No, newer models talk to themselves in this degenerated form of English all the time. It's a known problem that deep frying transformers with RL reduces CoT legibility, and the only known way around it is the Most Forbidden Technique - grading chains of thought.

▲

mietek 4 hours ago | parent | next [-]

Why is that not only forbidden, but also most forbidden?

	▲	erincandescent 3 hours ago \| parent [-]
		Because the CoT is supposed to be the independent control. If you RL on the CoT, then you lose it as a means of understanding the internal workings of the mdoel

▲

jephs 4 hours ago | parent | prev [-]

There's Most Forbiddenish and there's Most Forbiddener. We already apply length penalties to CoT in RL, for instance.

Sometimes I worry that the framing around CoT grading prevents research into necessary and useful safety techniques that edge up against it, but what do I know I'm just a guy, c'mon n downvote me LW bros

	▲	andai 28 minutes ago \| parent [-]
		Is the thing here that, we can get massive performance and efficiency gains by using vector space instead of English, but then we won't be able to know what's going on? Although, I thought we proved multiple times years ago that the model's decision-making process and reasoning tokens don't necessarily overlap?

▲

Starlevel004 5 hours ago | parent | prev | next [-]

Both Astra and Sol 6.1 do it, and Sol is even worse as it leaks through everywhere. Whenever it issues a subagent prompt it's just a total vomit of tokens mixed up together.

▲

sandos 3 hours ago | parent [-]

I sometimes find Sol 6.1 both printing random words, and producing code that is not super-dense, but far off from normal code with no empty lines and very logically dense.

It almost feels like its trying to protect its code from scrutiny, or like a developer who want job safety.

	▲	piperswe 2 hours ago \| parent [-]
		I've noticed it writing golfed code too. Maybe they're training it to avoid writing unnecessary whitespace tokens then run a formatter afterwards?

▲

yearolinuxdsktp 4 hours ago | parent | prev [-]

It constantly leaks into actual writing. I periodically have to correct Astra to fix lack of spaces, and sometimes, even outright dropping of intermediate tokens.