Remix.run Logo
lucideer a day ago

This is very doable - it's a shell you're creating within the cli so you have a lot of control over the environment. If you look at tools like sandbox-exec & apparmor, they specifically target binaries, not string input. E.g. if I write a bash script & put it in my path, OpenCode's allowlists aren't looking at the commands my bash script is exec-ing, nor monitoring subshells it creates. A proper sandbox has full context of the execution chain. Not only that, it's potentially much more powerful & expressive in that you can allowlist indirect filesystem access from commands run.

For example, I recently ran opencode via sandbox-exec & naively granted the sandbox access to the aws cli. First attempt to exec it failed because I hadn't granted the sandbox access to the credential file in ~/.aws. This took 2 seconds to resolve but effectively highlights how utterly useless the tree-sitter string-parsing they're doing is. It's so concerning because implementing tree-sitter for that is non-trivial to the point that any competent dev should surely realise quickly while implementing it that this avenue is utterly useless. Yet every high-profile agent cli has taken this approach.

qarl2 a day ago | parent [-]

Yes, I am aware that sandboxing technology exists. I use them. I don't expect them to be built into my LLM harness because I value modular software composition - and sandboxing should be provided by a product that specializes in that area.

OpenCode's allowlists are not there for security. They exist for LLM behavior guidance.

lucideer 20 hours ago | parent [-]

> allowlists are not there for security

I don't honestly believe that the allowlist isn't (naively) intended to be there for security. It's certainly not a very useful ux for providing LLM guidance: prompts, memories & AGENTS.md already provide an avenue for this & having it in a formalised, structured format isn't conducive to LLM guidance. The implementation brings in a lot of dependencies (tree-sitter) & contains enough complex targeted string parsing that's definitely superfluous for something intended to be fed to an LLM as guidance. Perhaps most importantly - the LLM behaviour on encountering limits of the allowlist is neither intuitive nor consistent enough to be considered reliably guiding (& system prompt doesn't do anything to mitigate this).

> I don't expect them to be built into my LLM harness because I value modular software composition

I value software modularity, in particular where software architecture is concerned, but I wouldn't go as far as being prescriptive about the unix philosophy. Direct integration does have its pros, & most of the cons associated with it in practice are specific to bad architecture (or black box software). For example Claude cli does integrate sandbox-exec internally, but it does do in a closed-source inaccessible way that negates its value. I am confident it can be done well though, even if I'm somewhat less confident that it ever will be.

42 minutes ago | parent | next [-]
[deleted]
qarl2 15 hours ago | parent | prev [-]

Well - pretty much everyone knows it's lousy security, because you know, it's pretty obvious.

But maybe some people think it's not obvious, and point it out because they think they're clever?

I suppose that's possible.