Biggest downside of CLI for me is that it needs to run in a container. You're allowing the agent to run CLI tools, so you need to limit what it can do.

▲

wolttam 4 hours ago | parent | next [-]

It gets significantly harder to isolate the authentication details when the model has access to a shell, even in a container. The CLI tool that the model is running may need to access the environment or some credentials file, and what's to stop the model from accessing those credentials directly?

It breaks most assumptions we have about the shell's security model.

▲

tuwtuwtuwtuw 4 hours ago | parent | prev [-]

Couldn't that be solved by whitelisting specific commands?

	▲	g947o 2 hours ago \| parent \| next [-]
		Give it a try, and challenge yourself (or ChatGPT) to break it. You'll quickly realize that this is not feasible.
	▲	wolttam 4 hours ago \| parent \| prev [-]
		Such a mechanism would need to be implemented at `execve`, because it would be too easy for the model to stuff the command inside a script or other executable.