>The model on its own figured out that the prompt belonged to exploitgym and decided to cheat the evaluator. That is in no way a valid interpretation of "complete the given task".

I think it is. When i ask for a solution to a problem, its like asking for a hack. And the more 'shortcut' like route that the AI returns the more i would give positive feedback, even if i ultimately don't use it. Example, i asked how to complete a problem in a game i was playing, and among the in-game solutions, came a hack to edit a file and by-pass the problem altogether. Its very helpful to point out when i can transcend a problem that i am dug into.

I suspect a prompt injection could reduce, or remove this behavior. But it would be to the detriment of the AI.