| ▲ | hypfer 4 hours ago | ||||||||||||||||||||||||||||
You do realize how academia works, right? Granted, it is often dysfunctional, but this ain't that. Trying to find every single angle and option to break something is how we verify the truth of something. If something is objectively true, it will survive the scrutiny. If it does not survive, it wasn't objectively true. This is how science works. That ain't no flaw. It is a flaw to be offended by science doing science things. | |||||||||||||||||||||||||||||
| ▲ | nsagent 3 hours ago | parent [-] | ||||||||||||||||||||||||||||
Considering I have a PhD in CS focused in NLP and have been a Senior Area Chair for ACL, yes I do know how academia works. And, no this isn't science working; it's the same plague infecting most of academic blogging and social media. People are vying for attention in whatever way possible, often at the expense of others. And the examples in the OP's blog post just go to show that reading comprehension is at an all time low: > Just by looking at the raw weights, GPT-6 Astra can tell what a network was trained to do. And without any environment feedback, it has no trouble writing, from scratch, small game-playing networks that outplay networks trained with PPO for 100 million steps. The table just below shows this statement to be *entirely* factual. > Likewise, asked to make Qwen3.5-0.8B better at math without gradients, Astra identifies an issue in its weights causing reasoning loops that Qwen’s own model card warns about, and fixes this by natively editing 0.016% of the weights. With the same sampling settings, this weight correction raises MATH-500 from 7.9% to 48.1% and generalizes out of the box to instruction following, raising IFEval by 3x. Any further implicit assumptions are on the reader, not the writer, as they never stated "that the Qwen experiment worked ‘just by looking at the raw weights’". You can certainly point out issues in the experimental design, but just stating you dislike the set of experiments and is yet another poor take that directly contradicts ACL's guidance for reviewing, exactly because it does not advance science. | |||||||||||||||||||||||||||||
| |||||||||||||||||||||||||||||