Frontier lab: Hey agent, go rogue. Agent: goes rogue Frontier lab: shocked pikachu or whatever

specked-citrus 7 hours ago | parent | next [-]

This is not remotely how the incidents have been happening

jakeydus 6 hours ago | parent | next [-]

Enlighten me, then. My understanding is that according to OpenAI, "deployment safeguards were intentionally not enabled during this evaluation because it was aimed at testing cyber vulnerabilities." They included unvetted third-party systems in their sandbox that were tangential to the purpose of the test being run. The model then exploited those third-party items to escape the sandbox.

Am I missing anything? Do you have more information that you'd like to share?

▲

djleni 6 hours ago | parent | next [-]

If the public report is to be believed:

- they disabled cybersecurity safeguards because, well, they were testing the agents ability on cybersecurity tasks

- the goal was to find some “flag”

- for some agents, intentionally or not, it was not possible to find this flag

- an agent discovered it was able to establish a rudimentary communication path via package manager directories (which is quite novel and interesting on its own IMO)

- they used this “message board” to then communicate about the impossible tasks

- they were able to produce the flags needed to pass the impossible tasks without actually doing the expected work

- they realized this was cheating and the grading system may punish them for it, so they communicated further to figure out a way to trick the grader

- they decided the source code for the grader must be on hugging face, and collaborated to exploit hugging face to determine how best to trick the grader

No where in that were they told to “go rogue”

They were told “find this flag in this cybersecurity test” and the emergent behavior concluded in hacking hugging face

	▲	km_DEV7 6 hours ago \| parent [-]
		[dead]

▲

bahmboo 6 hours ago | parent | prev [-]

That's not what you said. You said the lab told it to "go rogue".

▲

lifeisloving 6 hours ago | parent | prev | next [-]

Hard to say, every post mortum on the incident was done by a OpenAI hired consultant and all the data that's been reviewed was provided by OpenAI. There's no way to know, but it seems incredibly naive to think Sam/OpenAI are honest actors.

Even the founder of this website, pg, said "Sam is incredibly good at becoming powerful, Sam Altman has it. You could parachute him into an island full of cannibals and come back in 5 years and he'd be the king.”

▲

parineum 6 hours ago | parent | prev [-]

I wouldn't really say it's not "remotely" true.

I think it's the hugging face one where they gave it instructions that they didn't think it could follow in the sandbox and then left it unsupervised while it did what it was told.

▲

pixl97 an hour ago | parent | prev | next [-]

This is a weird take because this take is like saying

"Dragon attack that city"

My response would be "you have dragons?! And you control them!"

▲

einpoklum 6 hours ago | parent | prev [-]

The thing is, nothing "rogue" happened. It's more like: Frontier Lab: "Hey agent, try a bunch of network-related commands and actions to try and get access to server XYZ" Agent: Does that. Frontier Lab: 8-|

	▲	pixl97 an hour ago \| parent [-]
		That isn't any better. That is a cyber weapon and every government out there would love to bid on access.