If the code can call a method that provides the API key, what would stop the LLM from calling the same code? How do you propose to let an LLM run tests that execute code that requires API without the LLM also being able to grab the key?

I don’t give it access to calls requiring API keys in the first place.

This is just good dev environment stuff. Have locally hosted substitutes for everything. Run it all in docker.