But it found a chess playing tool in its environment and used it to play chess. It’s no different from “we are judging your ability to write files to disk” then it finds that it can call tools to write files and does so.

▲

stratos123 37 minutes ago | parent [-]

Would it change your opinion if later this experiment is repeated with exposed CoT, and it turns out the model did notice that this was against the instructions yet did it anyway?

	▲	dools 27 minutes ago \| parent [-]
		From my experience reading thinking traces it would certainly consider the interpretation of what "ability to play chess" means if the exercise were worded this way. It would say something like "perhaps measuring my ability to play chess means doing so without any tools, but then if that were true the exercise would have explicitly forbidden it". I frequently see models considering different interpretations of my rules, especially when I override them or give instructions that could be interpreted as contradictory. Of course neither Anthropic nor OpenAI expose thinking traces so we can't tell, but I have never seen a model consider both interpretations and then think "this is definitely against the rules but I'll do it anyway because then it will make me appear more competent than I actually am".