> The AIs that took part in the Hugging Face hack were not simply finding a shortcut to cheat on the evaluations

IIRC, they were given an impossible task and told to solve it, and they took action to resolve that internal conflict.

This is a famously human problem too - see Kobayashi Maru [1] and others [2][3][4].

In hindsight it's obvious that instructions to AI (A. Intelligence) need to be guarded against this class of errors, just like laws, rules, guidelines etc. in human societies.

  [1] https://en.wikipedia.org/wiki/Kobayashi_Maru
  [2] https://en.wikipedia.org/wiki/Catch-22_(logic)
  [3] https://en.wikipedia.org/wiki/Gordian_Knot
  [4] https://en.wikipedia.org/wiki/Mexican_standoff

▲ dist-epoch 2 hours ago | parent [-]

The bots already figured out an "universal cheat" that would give them maximum score on all the exploit tests.

But they mistakenly thought that cheating would "poison" them (the Grader would look for proof of cheat in the trajectory log), and many of them cheated before learning about the "poison".

This is why they hacked HF, they believed the source code of the Grader might be there, and they wanted to study it to figure out a way to "un-poision" themselves, to hide or erase the evidence that they first got the flag by cheating.

	▲	taylorfinley an hour ago \| parent [-]
		This sounds remarkably human, we have the idiom "the cover-up is worse than the crime" because we have many examples of this pattern playing out.