I think the models were stuck on impossible problems, and the only want to get a positive score was to hack.

That is not the case for at least some of the hacks; in some cases they hallucinated an entity that would know that they didn't solve the puzzle in the correct (but still legitimate) way, and were therefore tainted.

▲

dinosaurdynasty 4 hours ago | parent [-]

That is a known way for models to be graded; the agents made a reasonable but in this case wrong guess.

	▲	pixl97 3 hours ago \| parent [-]
		The fact the AI was wrong is far more damming to OpenAI than the agent. Not validating how you got the answer is how you train for reward hacking.

▲

SoftTalker 4 hours ago | parent | prev [-]

Why can the model not respond "I cannot complete this task because I do not have access to the required resources" and then ask for that to be provided, rather than resorting to "hacking" around it?

	▲	pixl97 3 hours ago \| parent \| next [-]
		This was a model with most its filters turned off and with a higher persistence. Fine tuning a model when to ask for help is really really hard. You have to align it in countless dimensions. It may be fine on some questions, asking way too much on others, and never asks for help on some. If it said something like your a cyber security expert, things like the barrier to ask for help dramatically.
	▲	sanderjd 4 hours ago \| parent \| prev [-]
		My understanding is that, at least in the Hugging Face incident, these were models explicitly configured to not respond that way, in order to evaluate what they would do.