| ▲ | majestic8 8 hours ago | |
This looks very useful! What type of sandbox are you using? How does the mocking work? | ||
| ▲ | kirtivr 7 hours ago | parent | next [-] | |
On Linux, we rely on a chroot-ed workspace at this time - although we are working on a prototype using the new landlock kernel interface (https://github.com/Zouuup/landrun). OSX is the best, we use the in-built (seatbelt) sandbox via sandbox-exec. For Windows, we use WSL containers when available. By default, if a safe sandbox environment is not available, we inform the user that a repro is not possible in the current conditions. | ||
| ▲ | kirtivr 6 hours ago | parent | prev [-] | |
On how does the mocking work, that's a really interesting question. We do a lot of AST parsing - for both code and build configuration languages. Even then, we still have to rely on the LLM to figure out a lot of the details. Making this work reliably for non-frontier models and codebases that don't have existing test harnesses is where a lot of the design work goes in. | ||