Remix.run Logo
▲ nsagent 3 hours ago

Considering I have a PhD in CS focused in NLP and have been a Senior Area Chair for ACL, yes I do know how academia works. And, no this isn't science working; it's the same plague infecting most of academic blogging and social media. People are vying for attention in whatever way possible, often at the expense of others.

And the examples in the OP's blog post just go to show that reading comprehension is at an all time low:

> Just by looking at the raw weights, GPT-6 Astra can tell what a network was trained to do. And without any environment feedback, it has no trouble writing, from scratch, small game-playing networks that outplay networks trained with PPO for 100 million steps.

The table just below shows this statement to be *entirely* factual.

> Likewise, asked to make Qwen3.5-0.8B better at math without gradients, Astra identifies an issue in its weights causing reasoning loops that Qwen’s own model card warns about, and fixes this by natively editing 0.016% of the weights. With the same sampling settings, this weight correction raises MATH-500 from 7.9% to 48.1% and generalizes out of the box to instruction following, raising IFEval by 3x.

Any further implicit assumptions are on the reader, not the writer, as they never stated "that the Qwen experiment worked ‘just by looking at the raw weights’". You can certainly point out issues in the experimental design, but just stating you dislike the set of experiments and is yet another poor take that directly contradicts ACL's guidance for reviewing, exactly because it does not advance science.

▲syllogism 2 hours ago | parent | next [-]

All I can say is I very much disagree, and if I had authored the post and it had received the reaction from people that it has I would be anxious to issue corrections. I haven't reviewed for conferences for several years now, but I wouldn't have thought this should pass review.

I believe the post definitely plays off the implication that the Qwen experiments are connected to the experiments that read the raw weights and have no Python harness, when this isn't the case. There are experiments where Astra reads weights directly for very simple models, and there are experiments where Astra performs ML research tasks by analysing the problem and writing Python code. The authors imply that the same technique underlies these but that isn't the case.

▲O_H_E an hour ago | parent | prev | next [-]

I mean separating two paragraphs by "Likewise" is pretty much how an author communicates to a reader to follow the barely-hidden line of reasoning.

▲hypfer 2 hours ago | parent | prev [-]

Link?

▲nsagent 2 hours ago | parent [-]

You have to be more specific. What exactly are you looking for a link to?