> A low probability thing when looking at how many human prisoners escape by talking a guard into just getting them out.

But this has actually happened... a lot. Search "social engineering prison breaks".

With AI it only needs to happen once.

I'm reminded of the scene in idiocracy where the protagonist, going through intake at the jail, tells the guard he's supposed to be getting out today, to which the guard says "you're in the wrong line dumbass" and waves him through.

To a true superhuman intelligence, we're the idiots who are theoretically easy to manipulate.

▲

RandomLensman 2 days ago | parent [-]

I didn't say it doesn't happen, but that it is a low probability. And we have ways to reduce probabilities in critical areas.

There is no omnipotent AI currently (and there might never be) and I don't see why with current AI it only needs to happen once.

▲

ben_w 2 days ago | parent [-]

They don't need to be omnipotent, and they're already human-or-superhuman at persuasion: https://arxiv.org/html/2411.06837v2

This may just be that humans find long arguments more persuasive than short ones, obviously LLMs can do that easily, but the outcome is I think more important than the mechanism.

▲

RandomLensman 2 days ago | parent [-]

That is about persuasion with evidence on various topics, not about persuading people to abandon safty protocols and processes and highly policed settings.

Yes, many things could happen, but again, that failure is possible is not a reason to do implement processes etc. I don't see why hypotheticals should stop addressing actuals.

▲

Smaug123 2 days ago | parent [-]

I’m afraid human red-teamers against supposedly highly secure targets, with lots of protocols in highly policed settings, do frequently manage this kind of social engineering. There’s loads of stories of pentesting military establishments, for example.

	▲	RandomLensman 2 days ago \| parent [-]
		Is there data on how frequently and what types of security levels? Military has varying levels of security and secrecy, for example. Also, not a reason not to pursue processes etc., no? I doubt that things fail all the time, for example.