Why not just do remote model isolation? Like remote browser isolation. Run your local model / agent on a little box that has access to the internet and also has your repository, but doesn't have anything else. Like BrowserBox.

You interact with and drive the agent over a secure channel to your local machine, protected with this extra layer.

Is the source-code the secret you are trying to protect? Okay, no internet for you. Do you keep production secrets in your source-code? Okay, no programming permissions for you. ;)

The easiest way to do that today is to use one of the cloud-based asynchronous coding agent tools - like https://claude.ai/code or https://chatgpt.com/codex or https://jules.google/

They run the agent in a VM somewhere on their own infrastructure. Any leaks are limited to the code and credentials that you deliberately make available to those tools.