This is exactly what I do - I run my agent inside an isolated podman container and mount the workspaces on a shared volume.