In practice, intelligence tests don’t depend on the specific questions asked. If you let a group of people generate their own questions, pool them together, sample randomly, and then rank scores, the same individuals would tend to rise to the top. This is because people with higher general cognitive ability perform better across virtually any cognitive task, a phenomenon first documented by Spearman (1904) and repeatedly confirmed in psychometrics research (e.g., Jensen 1998; Deary 2012).

Yes I'm aware of that, and I had this in mind when I wrote that "IQ tests measure the ability to solve quickly some abstract problems in an exam-like situation." Maybe that was not carefuly worded enough.

I am of the opinion that, although there is some generality in IQ tests, they measure only one specific aspect of cognitive ability defined by: abstract + quick.

My intuition is that cognitive abilities encompass much more than that. Anyone who have ever argued at length with that smart but obtuse engineer who can't tell the forest for the tree will know what I'm refering to. To me, a better test for "cognitive abilities" would also measure how someone is able of nuance, of humor, of seeing things from different perspectives, of introspection, etc, not just solving puzzles that can be described in a couple of sentences.

And I'm not talking about "emotional intelligence" here. To me, E-I is just the other side of that same flawed model that smells too much like modern day phrenology.