| ▲ | obblekk 3 hours ago | |
Yes training on math would give you spiky abilities on math. But in this case the dataset they measure perplexity is ood from training so lower perplexity on it indicates a general increase in intelligence across domains. But there’s definitely spikiness and it’s very possible they’ve trained more on math than other domains. | ||