The real solution to these kind of problems is sandboxing. I use podman through a bash script to launch a container whenever I want an agent to work on one of my repos. When done I just generate git patches and port back everything generated.

In this way I'm not afraid of letting the agents totally lose on my computer.