I can trivially write code that safely puts untrusted data into an SQL database full of private data. The equivalent with an LLM is impossible.

▲

dvt 6 hours ago | parent [-]

It's trivial to not let an AI agent use curl. Or, better yet, only allow specific domains to be accessed.

▲

strbean 6 hours ago | parent [-]

That's not fixing the bug, that's deleting features.

Users want the agent to be able to run curl to an arbitrary domain when they ask it to (directly or indirectly). They don't want the agent to do it when some external input maliciously tries to get the agent to do it.

That's not trivial at all.

▲

dvt 5 hours ago | parent [-]

Implementing an allowlist is pretty common practice for just about anything that accesses external stuff. Heck, Windows Firewall does it on every install. It's a bit of friction for a lot of security.

	▲	acjohnson55 3 hours ago \| parent \| next [-]
		But it's actually a tremendous amount of friction, because it's the difference between being able to let agents cook for hours at a time or constantly being blocked on human approvals. And even then, I think it's probably impossible to prevent attacks that combine vectors in clever ways, leading to people incorrectly approving malicious actions.
	▲	wat10000 5 hours ago \| parent \| prev [-]
		It's also pretty common for people to want their tools to be able to access a lot of external stuff. From Anthropic's page about this: > If you've set up Claude in Chrome, Cowork can use it for browser-based tasks: reading web pages, filling forms, extracting data from sites that don't have APIs, and navigating across tabs. That's a very casual way of saying, "if you set up this feature, you'll give this tool access to all of your private files and an unlimited ability to exfiltrate the data, so have fun with that."