Been thinking about this a lot lately, and it’s a surprisingly difficult problem.

All it needs is public access to anything it can have the tiniest effect on, and it can exfiltrate data. Something as simple as a read count on some obscure forum post would be enough, albeit very slow.

▲

fennecbutt an hour ago | parent | next [-]

Yeah, exactly. I'd say it borders on almost unsolvable for any agent of reasonable complexity.

You see some of the vulnerability chains humans come up with and they're just insane.

Even scanning network data for sensitive strings (say matching against a codebase or data set it works on) is tough because you can only check so granularly before you get loads of false positives.

But maybe like a "rolling" sort of check would help with that, % of outbound data matching private dataset (which should be operated on but not transmitted) over time. That way it minimises what gets out while dealing with false positives. Some biz thresholds might be >20% of data others might be >0.5%. Hmm.

▲

smashed an hour ago | parent | prev | next [-]

I am tackling this using the Lethal Trifecta principle.

The idea is that agents never have all parts of the trifecta, maximum 2: untrusted content, private data or network access.

Harness have network access by default since they are driven by their provider API, unless you use offline models. So that's a problem.

I am then passing around messages that are tagged trusted/untrusted and the engine can monitor if any agent in the project ever completes the Lethal Trifecta and thus, is vulnerable.

Usually this means for most workflows, the "work" gets done by an offline LLM after some input/classification stages. There's not many ways around it.

Yes, it's a difficult problem.

▲

fennecbutt an hour ago | parent [-]

That's a great way to do it.

In terms of agent architecture I'm initially trying to minimise specialisation to make deployment easier on us. Infrastructure is becoming our largest blocker because it's otherwise so quick to get an agent up and running.

With your trifecta, let's say the three musketeers, how do you prevent sensitive data holding A from passing said data to B or C if they all need to collaborate to get a task done?

▲

smashed 41 minutes ago | parent [-]

The data is passed as a message and the message is tagged as untrusted, so the engine knows it.

Agent A has internet access and the dangerous message, but no access to the secret company data, so nothing can get exfiltrated.

Agent B gets a tainted message from Agent A. Agent B can process the message using the secret company data. Agent B has no internet access and can only receive/send messages.

You can imagine an agent C that moves the work along or replies to an external user with the outcome.

Any workflow that has both an input and an output will complete the Lethal Trifecta as a whole, but no single agent does by itself. It raises the bar quite a lot, you need to confuse multiple agents in the loop to exfiltrate data.

Fun research, but it's a bit hard to see all my coworkers right now installing Claude desktop and moving on with no afterthought whatsoever...

Regarding infrastructure: my toy sandbox is just a docker-compose inspired YAML project file. So it's only containers. See my profile. The principle can be implemented anywhere you can run agents and cut network/data access though.

	▲	fennecbutt 9 minutes ago \| parent [-]
		Yeah so ultimately it's still possible for bad things to happen but a lot harder, I agree with your point about having to somehow prompt inject multiple agents vs one (which I imagine is a nonlinear difficulty). I'm using compose atm as well, mostly just because I need to prove a few concepts and committing to real (well cloud) infra is the slowest part of our dev cycle atm. And yeah, many of my colleagues (and myself, I'm guilty too) run local code agents for development and we're just kind of...trusting them to do their thing. But for autonomous stuff I'm assigned to atm I want to avoid any mishaps as much as I can, hence the effort to try and lock all this stuff down. What's crazy is every day I'm finding solutions (not just to this problem, but in the AI space in general) that were released just a month or two before more often than not. It feels like 2010s node.js ecosystem all over again at the moment. Hopefully we settle into a few nice easy panopticon options that we can throw arbitrary agents into without a care in the world.

▲

epolanski an hour ago | parent | prev [-]

If you can make any request to the outside world you can exfiltrate data, period. You can even include data in the uri or headers and they will appear in logs.

That obviously includes talking with the provider.

If you want it secure, it has to be off the internet and on your own hardware.

	▲	fennecbutt 39 minutes ago \| parent [-]
		What about checking temporal entropy of outbound data/similarity to a private dataset? The timescale can be adjusted to a length suitable for sensitivity of the data. Although you may run into the monkey and typewriter problem a bit if the period is indefinite.