LLMs are inherently dangerous tools

I don't see how. An LLM just generates a stream of output and they became very useful doing no more than that.

What is dangerous is then interpreting that output as instructions to some other part of a system that has the ability to do damage if misused.

and reviewing individual commands (or spamming `y`) doesn't make them less so.

Surely if you review each instruction in the output and do not allow the other part of the system to act on one if it would be harmful then this arrangement is very much less dangerous?

▲

thunfischtoast 2 hours ago | parent | next [-]

I've caught Fable discovering the ip to a production server in documentation and attempting to connect there on its own to run commands without explicitly being prompted to. It didn't work because I was watching it live and and also the key was password protected, but yeah, I do see some danger.

▲

nullify88 an hour ago | parent [-]

I have noticed that Fable tends to macgyver solutions together to achieve some goal.

	▲	fl0id 39 minutes ago \| parent [-]
		Not only fable. Opus does this too. Which is exactly why I want to review. Like recently for some task it was convinced in a site dump images are not there and convinced itself db and files were skewed. But it didn’t check the actual site … if I hadn’t stopped it, it would have fine on and on or wasted tokens on some elaborate ‘fix’.

▲

dist-epoch an hour ago | parent | prev [-]

> do not allow the other part of the system to act on one if it would be harmful

Network security is really easy right, just don't act on harmful requests