Remix.run Logo
nextaccountic a day ago

Elsewhere in the thread someone linked to https://pleasedonotescape.com/ which is a list of sandboxes

Maybe sandboxing isn't the job for an agent harness?

Maybe choosing a VM as your sandbox, with access only to project directory.. and then enabling YOLO mode on the agent, is the way to go.

And in any case, for an agent that can write and run code, no allowlist for which programs are allowed to run is sufficient, since the agent can just write the code the program would run anyway. And the issue isn't which software the agent runs, but whether they inherit the same ambient authority as your user - read the same files as you do, modify your config files, etc. Running in a VM stops that.

Clawk (recently submitted to HN here https://news.ycombinator.com/item?id=48892859) might be good

lucideer a day ago | parent [-]

I've run agentic CLIs in a sandbox (using MacOS sandbox-exec) & in VMs - both solutions have an inherent advantage over any integrated sandboxing by virtue of their independence (even before we get into configurability). Claude CLI integrates sandbox-exec internally, but I can't really trust it completely because Anthropic don't really have any vested interest in restricting their own tooling on my machine - they want me to rely on the tooling more, not less, which naturally entails granting the tool broader access: this is reflected in two ways in their sandbox-exec implementation: (1) it's wrapped in a relatively lax default profile & (2) not everything the cli does is sandbox-wrapped.

All that said, I don't think I'd agree that sandboxing doesn't belong in the agent harness. Ultimately, I want to be using an open source cli tool that doesn't come with the above Anthropic perverse incentives. The advantage of using a separate sandbox solution with Claude CLI isn't that it's separate, it's that I control it. Ideally I should control my agentic cli though, & in that context, bundling the sandbox comes with a lot of inherent advantages.

Unfortunately, as far as I've seen, the current batch of open source clis are pretty poor at the above, leaving little incentive to switch from proprietary solutions.

nextaccountic a day ago | parent [-]

I really, really wanted to make agent sandboxes work. Filed issues on the sandbox created by Anthropic and later also adopted by Copilot, and also to Codex repository. I think none of them is actually usable - they either allow too much or too little.

There is one case where VMs are not fit, and it's when you need to do GPU work. Well unless you have gpu passthrough or virtual gpus.

lucideer a day ago | parent [-]

I've used VMs because dealing with tool access is infinitely simpler & less fraught than tailoring sandbox profiles, but ultimately using sandbox-exec has yielded better workflows (if & when I've gotten an appropriate profile stood up). The sandbox tends to have a lot more rough edges than the VM, so I do switch back & forth between VM & sandbox, but I suspect a good sandbox-wrapping implementation with nice ux is the future.

Perhaps one could go further & combine them: e.g. a sanbox-wrapped on-host CLI that subshells/subagents out to VMs on demand. Seems potentially overcomplicated but I guess one can dream.