why assume that because LLMs are strong in a narrow set of fields it will be strong in everything? we don't make that assumption about other forms of intelligence. i.e nobody thinks Stockfish is good at product management, and people generally don't think that a genius composer will be equally good at botany.

▲

famouswaffles 7 hours ago | parent | next [-]

In 1904, Psychologist Charles Spearman noticed something interesting about school children's academic performance. Students who excelled in one subject tended to do well in others even when those subjects appeared quite different. He proposed that a common underlying ability, which he called the general intelligence factor, or g, could explain these correlations. [1]

Today, g is one of the most robust findings in differential psychology. The tendency for different cognitive abilities to correlate positively has been replicated across numerous studies. g typically accounts for around 40 to 60% of the variance in cognitive test performance and is predictive of numerous life outcomes, including educational achievement and occupational performance. [2]

Evidence of g isn't confined to humans either. Studies of mice have identified a general cognitive factor explaining roughly 30 to 40% of the variation in performance across different learning tasks [3]. We have similar results for primates and birds.

To put it simply, intelligence tends to generalize. Humans who tend to have higher verbal skills also tend to have higher spatial skills, better memories, and faster processing speed. Someone with the cognitive ability to become an exceptional chemist might just as easily have become an exceptional mathematician or software engineer. The knowledge and skills required are obviously different, but the underlying cognitive abilities that make someone successful in one intellectually demanding field often transfer to others.

And yes, there is also evidence of g in LLMs [4].

[1] https://www.jstor.org/stable/1412107

[2] https://pmc.ncbi.nlm.nih.gov/articles/PMC8293439/

[3] https://pmc.ncbi.nlm.nih.gov/articles/PMC2614349/

[4] https://www.sciencedirect.com/science/article/pii/S016028962...

	▲	slopinthebag 6 hours ago \| parent [-]
		cognitive abilities correlate across a particular set of measured tasks but it's not established that high performance in those tasks generalises to arbitrary domains theres a difference between having a general capacity to learn, possessing domain-specific expertise, and being able to reliably apply that expertise in the real world. even in humans, a genius mathematician isn't necessarily an exceptional manager, composer, or biologist. general intelligence might make it easier to acquire those skills but it doesn't substitute for them. the question isn't whether LLMs exhibit "g" but if the abilities being measured are representative of the broader capabilities we're predicting. that's not something g settles.

▲

perching_aix 10 hours ago | parent | prev [-]

they did not specify LLMs

	▲	slopinthebag 8 hours ago \| parent [-]
		they are clearly talking about llms