I’m surprised the score isn’t higher. What’s to stop an LLM from training on the complete corpus of IQ tests. I assume they’d get perfect scores

I was thinking that too. I wouldn’t even trust that the “offline” tests didn’t have the questions and answers posted online somewhere. This might really be an analysis of how extensive the dataset is for each LLM, not how much smarter one LLM is from another.