That seems intuitive to me, but lots of other things in science seemed intuitive because I wanted to believe them. If the measured IQ difference in individuals can be overwhelmed by simple factors like “have I taken the test before”, we don’t really have a useful empirical measurement to say these things and we’re just stating our hopes and dreams.

▲

nialse 4 days ago | parent [-]

The training effect in test-retest is dependent on g as well. It is intelligent to learn from past experiences.

Measuring g is hard and taking shortcuts is tempting. A reasonable repeatable g factor test takes hours, and is too often replaced by a single test. There are ways around the test-retest issues but they are roads less travelled.

▲

thechao 4 days ago | parent [-]

My high school was right across from a branch of a university (UHD) where the PhD candidates developed IQ tests. We (the HS students) could take them for extra credit. My favorite example was a block-arranging test (there was a set of blocks & some pictures). Anyways, they printed the blocks "symmetrically"; once I figured that out, making the picture was limited only by how quickly I could move. (The test normally had you looking at all sides of the cube, repeatedly.) My "IQ" was well over 200 on that test. The candidate said that it was going to set their lab back bag years.

▲

mieubrisse 3 days ago | parent [-]

A similar thing happened to me.

I once took a timed test with a section that had me translating a string of symbols to letters using a cipher, response being multiple choice. If you read the string left to right, there were multiple answer options that started with the same sequence of letters (so ostensibly you had to translate the entire string).

But if you read the string right to left, there was often only one answer option that matched (the right one). So I got away with translating only the last ~4 symbols, regardless of how long the string was. I blew through the section, and surely scored high.

I always wondered: did they realize this? Or did it artificially inflate my results?

And looking at the highest-entropy section felt natural to me, but only because of countless hours as a software engineer where the highest-entropy bit is at the end (filepaths, certain IDs, etc).

Is it really accurate to say I'm "more intelligent" because I've seen that pattern a ton before, whereas someone who hasn't isn't? I suspect not.

	▲	nialse 3 days ago \| parent [-]
		If the pattern generalizes to other tasks, maybe the test was right? ;) Appreciate your post and the post you commented on. Taking shortcuts in test development often ends up being detrimental. There is also an inherent challenge in developing test for people who may well be smarter than you are. It’s like that programmer thing: “If you write the smartest program you can, and debugging is harder than writing code. Who’s gonna debug the code?” Many people have tried developing “smart” tests for cognitive abilities, some realize when they fail, some unfortunately don’t.