I agree. I wonder what the human baseline is for ”what is 1 + 1” on Rapidata.
We try a bit harder than that my friend.
I actually didn't mean to criticize Rapidata. I just think that a forced-choice question like this begs for low-effort answers. At least the respondents should have had the opportunity to explain their reasoning, like the LLMs did.