Remix.run Logo
▲ bigmadshoe 2 hours ago

> Given that the main way new surprising capabilities emerge is by progress on compression

Is it just me or does the article completely lack proof of this premise? I see a plot of perplexity dropping over time, with the implication that performance has risen as a result. This is just a correlation. What else did they change about post-training during this timeframe? This doesn't feel like a real experiment or real science.