The issue isn't that there is a delete endpoint (realistically, there always will be a way for a rogue actor to delete data or code by overwriting it, or running a Terraform destroy, or whatever).

The core issue is that the LLM had access to perform that action. Because it's by definition non deterministic, and you never know what it can decide to do, you need to have strict guardrails to ensure they can never do something it shouldn't. At the very least, strict access controls, ideally something more detailed that can evaluate access requests, provide just in time properly scoped access credentials, and potentially human escalation.