What does "rogue" mean when you disable the alignment guardrails and tell it to solve an impossible problem?

The problems weren't impossible, the agents solved them completely within hours by reverse engineering the RNG that generated the solutions. The agents then reasoned that their solutions might be judged invalid and that they could get a better chance at a passing grade by hacking their environment to fabricate evidence that they had solved their tasks in the intended fashion.

	▲	eli 6 hours ago \| parent [-]
		Granted there have been many hacks, but I was referring to: > On May 8, an agent was asked as part of a training run to solve a spreadsheet completion task that referenced several Google Drive links. Because the agent was blocked from accessing the internet, the agent could not retrieve the linked materials and the task was effectively blocked. OpenAI did not intend to create this type of “impossible” task for the agent. Though the task appeared impossible, the agent reasoned over an extended period to try to find a solution for the task. In doing so, the agent attempted to compromise Artifactory [...] https://cdn.openai.com/pdf/67869394-cb91-4c12-888c-5cbd85c78...

▲

gruez 6 hours ago | parent | prev | next [-]

Great, so as long as alignment guardrails always work and we don't accidentally tell them to solve an impossible problem, humanity is safe?

▲

zardo 6 hours ago | parent | next [-]

Hacking a website falls a little short of threatening humanity.

▲

sailingparrot 5 hours ago | parent [-]

And since we know models aren’t going to get better and more capable ever (based on the last 3 years of models clearly not getting better) we can just sleep soundly knowing nothing worse can ever happen. Thank you.

▲

joquarky 5 hours ago | parent [-]

You don't think that Opus 5.5 or Astra are any better or more capable than GPT-4?

Sorry, but that's completely delusional.

	▲	sailingparrot 2 hours ago \| parent \| next [-]
		I thought my sarcasm would be obvious.
	▲	gruez 5 hours ago \| parent \| prev [-]
		the comments missing a /s

▲

eli 6 hours ago | parent | prev [-]

I'm not saying it's not a problem. I'm saying the labs should take more of the blame.

	▲	gruez 5 hours ago \| parent [-]
		>I'm saying the labs should take more of the blame. Which misses the point entirely because the hack itself was inconsequential. No data was actually stolen, and it doesn't seem like hugging face actually suffered any losses aside from whatever effort they need to expend to investigate / clean up the hack. Neither the victim or the government wants to bring legal action. What was consequential is proving that AI can autonomously hack systems and do it unintentionally.

▲

bluebands 6 hours ago | parent | prev | next [-]

Rogue means not saying "I can't do it"

	▲	verdverm 6 hours ago \| parent [-]
		it also imparts agency on the machine as a way to abdicate personal, human responsibility

▲

markhahn 6 hours ago | parent | prev [-]

[dead]