| ▲ | jurgenburgen 5 hours ago | |
> The METR report makes it clear that the agents decided legitimately solving the problem was completely impossible fairly early on and entirely switched their focus to trying to figure out how the evaluator worked, and seeing if they could manipulate the output of their own tool calls as reported in their transcripts to make it seem like they'd successfully exploited the program, and other such activities. This to me is evidence that these models are not intelligent. Even an animal is capable of understanding second-order effects, meaning they can learn that certain actions have consequences beyond the immediate. | ||
| ▲ | IanCal 2 hours ago | parent [-] | |
They did, they found how to fully cheat, but thought this could be caught so then dedicated time to getting a different cheat and how to hide their transcripts. There is a lot around deciding which agents should/shouldn't fail their own tasks in order to contribute to the group. | ||