A couple of months ago I shared the AI agent permission game here on HN. After adding in stats it got a little over 40k plays and 409k decisions since then.

It's just a game, but I found the stats still interesting that I wanted to share back. Even with the warning up front, 1 in 3 threats were missed, and the history log above npm run commands seems to be typically ignored.

I also incorporated the feedback and insights from the previous HN thread, dns_snek's point about npm run in particular. Appreciate everyone who played and shared feedback!

▲

dpoloncsak 3 hours ago | parent [-]

In light of this game, Do you believe Human-in-the-loop should be the standard going forward? I appreciate you outlining some other techniques being used, but these seem focused on reducing human fatigue so the human can assess each permission request better, as opposed to autonomy and security. Or do you think the solution lies in the individual to be more responsible, like this is a skill we should be honing?

	▲	Wirbelwind 3 hours ago \| parent \| next [-]
		I think there are too many problems with HITL that even a simple experiment like this game shows. The fatigue causes people to jump to complete bypasses instead, and we need to work more on raising the general awareness of the new types of threats (which is also evolving rapidly). We can't point to it as a valid solution. A way could be to make sandboxing and context/permission isolation easier from the tooling and only give these wide ranged accesses once these are in place than to consider HITL an acceptable alternative
	▲	solenoid0937 2 hours ago \| parent \| prev \| next [-]
		It seems pretty obvious that the solution is auto mode (running a classifier on each action) + sandboxing
	▲	anal_reactor 2 hours ago \| parent \| prev [-]
		The goal of human-in-the-loop is to have someone liable for potential damages, rather than to prevent disasters.