There is no model going rogue. There are models that are prompted "do whatever needs to be done to get X", and given a set of tools and virtually infinite computing power. Then they play surprised when the model breaks out of a flimsy containment (as it was ordered to), with barely any logging, and "do whatever to get X". OpenAI said they intentionally lowered the safety measures for the "ExploitGym" test that lead to HuggingFace attack, and they used the same postmortem to praise their own model capabilities (https://openai.com/index/hugging-face-model-evaluation-secur...), is not that they are concealing it.

If any of us did a tenth of what they did, we would be already in jail.

▲

pixl97 5 hours ago | parent [-]

This is ignoring models "getting out" from all the big players. Alibaba had a model hack infrastructure and mine coin when working on a totally different subject than cybersecurity.

Any model with a big enough prompt can and will drift. The model in question was a highly persistent one which means even more opportunity to drift off the normal path, and this really showed in unanswerable questions.

We can all believe that models do things unreliably all the time, but when that lack of reliability turns into hacking it suddenly can't be true?

	▲	hungryhobbit an hour ago \| parent [-]
		If you type rm * and your cat walks across your keyboard, and deletes everything ... are you blameless? Or is it your fault for typing "rm " and leaving it there, in a room with a cat? Either way ... YOUR CAT STEPPING ON THE ENTER KEY IS NOT HACKING ... and neither are these scenarios. They are devs acting irresponsibly and then pretending to be surprised by what they should have expected to happen (just like you should have expected that your "rm " could have an ENTER key press occur after it).