Remix.run Logo
▲ obblekk 3 hours ago

Yes training on math would give you spiky abilities on math. But in this case the dataset they measure perplexity is ood from training so lower perplexity on it indicates a general increase in intelligence across domains.

But there’s definitely spikiness and it’s very possible they’ve trained more on math than other domains.