| ▲ | eulgro 3 hours ago | |||||||
Didn't they hit a data wall roughly two years ago and had to rely on artifical datasets for pretraining i.e. LLM-generated? I would expect the entropy of artificial datasets to be lower, thus easier to predict and scoring better on "perplexity"? | ||||||||
| ▲ | chubot 3 hours ago | parent | next [-] | |||||||
I thought that too, but models are trained on observation now, to increase agent performance. This is new data that is separate from Internet crawls. I think it's done with reinforcement learning with verifiable rewards https://github.com/opendilab/awesome-RLVR Examples: - When playing Go or chess, nobody may have written or published web pages with a particularly clever end game, but an agent can use CPU time to discover these games themselves, and learn from them. Human Go players report learning strategy from what AlphaGo discovered. - Agents use Unix shell and are trained with it. So they can discover things and combine things that nobody has written about on the Internet, like how to debug a rarely used driver, or an interaction between two drivers. And they can see patterns in all the drivers that humans may have a hard time even seeing, leaving aside the motivation to publish on the Internet. This explains why the performance is so "jagged" - great on coding, math and closed games, but less good on things involving the real world. We probably did hit a data wall, but then we got new sources of data. | ||||||||
| ||||||||
| ▲ | jrflo 3 hours ago | parent | prev | next [-] | |||||||
Yes, they started to hit a data wall, but I thought that much of the progress of the last couple years came from reinforcement learning after pretraining, which is mostly building complex environments for the models to reason in, rather than accumulating piles of raw text. | ||||||||
| ▲ | redox99 2 hours ago | parent | prev [-] | |||||||
I'd hope they measured PPL on the same dataset for all those models. And one that's fair and doesn't benefit some models more than others (probably non synthetic dataset). | ||||||||