Remix.run Logo
kingstnap 18 hours ago

The appeal of auto mode is obvious.

Non yolo mode where you manually approve each command is literally pure security theatre. No human on earth has the patience to discerne the huge stream of commands agents run.

So even if false negatives are technically zero, having 100% false positives is not acceptable.

So a fundamental premise is you have to have filtration. Where only very few things are surfaced for humans to look

The first option which a decent number of harnesses do is being able to setup an auto approve list. Like `ls` is fine, `cat` is fine etc.

Now maintaining this list is in and of itself a giantic pita. But the real issue is that it still has way too many false positives. Fundamentally it comes down to the halting problem where you can't really include important things like `bash python <<PY` and what not which agents like to use. But regex can't solve the halting problem to figure out if the Python is safe.

So naturally the next best option is to use an LLM. Which isn't that stupid because even if its non deterministic at least it can dramatically reduce the false positives from the regex auto approve list.