Remix.run Logo
▲ sfink 41 minutes ago

I think it's confusing and incomplete, but not wrong. Paragraph 2 says the Nov 1 test included 20 of the tasks and ran each twice. That's the 40 denominator.

Why paragraph 1 even mentions the full set of 30 tasks if 10 of them are irrelevant to the comparison, I don't know.

I'm not sure if it's LLM text, or somebody who has been reading way too much LLM text and is now corrupted into mimicking it. My guess would be a mix of human and LLM. "...evaluated by tests the agent didn't see" sounds very Claude. The first line (2 sentences) sounds human to me. Everything after sounds AI, though the last line is ambiguous.

It's kind of sad how our meat brains are getting non-consensually trained on this categorization task....