Remix.run Logo
mieubrisse 3 days ago

A similar thing happened to me.

I once took a timed test with a section that had me translating a string of symbols to letters using a cipher, response being multiple choice. If you read the string left to right, there were multiple answer options that started with the same sequence of letters (so ostensibly you had to translate the entire string).

But if you read the string right to left, there was often only one answer option that matched (the right one). So I got away with translating only the last ~4 symbols, regardless of how long the string was. I blew through the section, and surely scored high.

I always wondered: did they realize this? Or did it artificially inflate my results?

And looking at the highest-entropy section felt natural to me, but only because of countless hours as a software engineer where the highest-entropy bit is at the end (filepaths, certain IDs, etc).

Is it really accurate to say I'm "more intelligent" because I've seen that pattern a ton before, whereas someone who hasn't isn't? I suspect not.

nialse 3 days ago | parent [-]

If the pattern generalizes to other tasks, maybe the test was right? ;)

Appreciate your post and the post you commented on. Taking shortcuts in test development often ends up being detrimental. There is also an inherent challenge in developing test for people who may well be smarter than you are. It’s like that programmer thing: “If you write the smartest program you can, and debugging is harder than writing code. Who’s gonna debug the code?” Many people have tried developing “smart” tests for cognitive abilities, some realize when they fail, some unfortunately don’t.