Remix.run Logo
▲ stymaar a day ago

Tangent: LLM have stopped being stochastic parrots since RLVR was introduced in their training pipeline.

▲GMoromisato a day ago | parent | next [-]

Probably right, but I think chain-of-thought has had a bigger impact on intelligence in LLMs. I.e., LLMs can reason their way to a solution instead of trying to one-shot it from training data.

▲stymaar 18 hours ago | parent [-]

I'd argue that it's the same thing. Asking a language model to “reason step by step” only marginally improved the results before we steered the thinking traces with RLVR.

▲bigstrat2003 a day ago | parent | prev [-]

They have never stopped. It is inherent to the design of how they work. People just have bought into the hype trying to distract you from what they are. That doesn't mean they are useless tools, but don't ever make the mistake of believing they can actually reason.

▲CuriouslyC 20 hours ago | parent [-]

The point of stochastic parrots is they're just repeating things they've been pre-trained on, but RLVR pipelines teach the model to be good at "trying stuff" as part of an ad-hoc search algorithm.

▲jdub 19 hours ago | parent [-]

"stochastic parrot" is a metaphor, not an analogy

RLVR isn't magic, it just bends some numbers

chain-of-thought isn't magic, it just bends some numbers

▲stymaar 18 hours ago | parent [-]

> “stochastic parrot" is a metaphor, not an analogy

It used to be an apt description, but not anymore.

> RLVR isn't magic, it just bends some numbers

So is all machine learning, but it turns out if you bend them well enough you can get very powerful capabilities.

And also that's enough to say they aren't just trained on repeating what's in their training data, which is the reason to call them “stochastic parrots” in the first place.

And their capabilities have vastly improved since we've started doing it.

▲GoblinSlayer 7 hours ago | parent [-]

So can they now learn from each other?

▲stymaar 7 hours ago | parent [-]

Depends what you call each-other, but kind of: at least they learn from the failures and sucesses of their ancestor during the RL training pipeline.

(Like how AlphaGo learned to play Go in a way that no human ever did, just by playing against “itself”).