Remix.run Logo
simonw 4 hours ago

This looks really good - the CLI interface design is solid, and I especially like the secrets / network proxy pattern - but the thing it needs most is copiously detailed documentation about exactly how the sandbox mechanism works - and how it was tested.

There are dozens of projects like this emerging right now. They all share the same challenge: establishing credibility.

I'm loathe to spend time evaluating them unless I've seen robust evidence that the architecture is well thought through and the tool has been extensively tested already.

My ideal sandbox is one that's been used by hundreds of people in a high-stakes environment already. That's a tall order, but if I'm going to spend time evaluating one the next best thing is documentation that teaches me something about sandboxing and demonstrates to me how competent and thorough the process of building this one has been.

UPDATE: On further inspection there's a lot that I like about this one. The CLI design is neat, it builds on a strong underlying library (the OpenAI Codex implementation) and the features it does add - mainly the network proxy being able to modify headers to inject secrets - are genuinely great ideas.

kjok 4 hours ago | parent | next [-]

> There are dozens of projects like this emerging right now. They all share the same challenge: establishing credibility.

Care to elaborate on the kind of "credibility" to be established here? All these bazillion sandboxing tools use the same underlying frameworks for isolation (e.g., ebpf, landlock, VMs, cgroups, namespaces) that are already credible.

simonw 4 hours ago | parent [-]

The problem is that those underlying frameworks can very easily be misconfigured. I need to know that the higher level sandboxing tools were written by people with a deep understanding of the primitives that they are building on, and a very robust approach to testing that their assumptions hold and they don't have any bugs in their layer that affect the security of the overall system.

Most people are building on top of Apple's sandbox-exec which is itself almost entirely undocumented!

kjok 3 hours ago | parent [-]

> The problem is that those underlying frameworks can very easily be misconfigured.

Agreed. I'm sure a number of these sandboxing solutions are vibe-coded, which makes your concerns regarding misconfigurations even more relevant.

afshinmeh 4 hours ago | parent | prev [-]

Simon! Thanks. I appreciate your comment and totally agreed. I will improve the docs as well as tests.