I recently got my feet wet with Nix for the purpose of agent sandboxing, and it's a really great concept!

The issue I have run into is that for running llama.cpp, which is a very actively developed bleeding-edge software, it seems like experimenting with different versions/configs/patches etc. was fighting with the Nix philosophy of immutable software, so I ended up just managing it outside of nix.

But this could be user-error, I've only spent a couple of weeks with it.

> The issue I have run into is that for running llama.cpp, which is a very actively developed bleeding-edge software, it seems like experimenting with different versions/configs/patches etc. was fighting with the Nix philosophy of immutable software

May be. I'd guess it probably does it in a high-friction way compared to what you expect.

e.g. in a typical linux distro, you can just replace the (globally installed) version of the package and use that.

With Nix, you've got a declaration of how to build a package, including its dependencies; and each of those dependencies is also a declaration of how to build that package, and so on. -- And so, "just change this version" results in rebuilding the full chain of dependencies.

If you want multiple readily-available versions of llama.cpp each with different configs/patches, then using Nix would make this easier I'd think. But for "I just wanna try this, then try that", it's going to add overhead.

one can use direnv, nix-direnv and flake.nix to get an immutable and reproducible build toolchain protected from nix garbage collection, while keeping the llama.cpp branches and/or compiled binaries mutable on the local filesystem. https://github.com/nix-community/nix-direnv

for agentic execution sandboxing, preventing 'rm -rf /home' nix or nix-direnv doesn't help, podman could be used next to nix. https://discourse.nixos.org/t/what-is-sandboxing-and-what-do...

llama.cpp has a flake in-repo that I often use for trying out different branches and patches. I also have yet to have an issue just replacing the src and build number attribute in the nixpkgs build (in one cases I wanted to add an additional CMake flag but that was also easy).

IMHO this is way easier than without Nix in many cases as figuring out the upstream build process and getting it running can often be quite the chore. With Nix it is all set up for you.