They aren't. Allowlists are a deterministic tool. That's the whole point: instructing an llm to review itself is nondeterministic, and thus not a good tool for permissions, compared to allowlists, which are.

Yes, but we are not restricted to agentic interaction only. We can always pass our inputs and outputs through deterministic filters (which is sort of the point of MCP servers and tool calling, for some use cases).

Yes. Leaves a lot to be desired UX wise, has no good defaults, and still allows many holes that you need to dig to know (e.g. its own write tools vs bash).

Built my own with Apple's (to be "deprecated") seatbelt backend, which I think anthropic also uses, with a much nicer interface, easy per working project setup, etc.

What?

True... until their intentions are good enough.

I spent a long time as a younger man running general software inside ephemeral VMs. The UX was terrible, but I had no fears about browser vulnerabilities ruining my day. Ephemeral is hard, and the UX is still poor, but I did try it again a month or two back, and it's not impossible.

But now I am doing all my software development back inside ephemeral VMs, AI or otherwise - the UX there is much better, I significantly mitigate supply-chain attack risks, and agents are already constrained to a secret-less, low privilege environment, should I use them. There's a significantly higher likelihood of rogue software in third party dependency trees than there is in a small subset of programs you regularly use.

You could in theory fully review Claude Code.

But even that wouldn't be enough because LLMs are nondeterministic, and these kinds of issues are the inevitable results.

Only a mechanical, deterministic failsafe can protect you from it.

Text in a model context is not a reliable constraint.

I'm not claiming auto is worse than manual; I'm claiming it cannot be compared with anything security-wise because it does not aim to provide any guarantees, there's no systematic way to reason about what it will and won't allow, etc

With e.g. VM or container sandboxing, we can say things like "If there are no vulnerabilities in the hypervisor or hardware, then the agent can only access these files, these network services, etc". Auto mode doesn't even try to provide any guarantees, so it's impossible to reason about. It amounts to just using the models judgment about what's "malicious" or not, a definition which is subjective, changes model to model, and can be influenced by attacker input.

Empirical evaluations of how many vulnerabilities manual vs auto mode let through are not super meaningful because auto mode changes dramatically with model releases, the type of task being performed, whether the attacker is making targeted attacks designed to bypass auto mode, etc. A security feature cannot rely on these sorts of factors if it's to be taken seriously.

Further, percent of malicious tool calls accepted is a bad metric. It's like measuring the security of ssh in terms of what percent of unauthenticated logins succeed. The only acceptable value is zero. It's pass/fail, not a spectrum. So both auto and manual mode should fail equally here. Further, the set of malicious tool calls admitted in auto mode may not be a subset of those admitted in manual mode.

As real as approval fatigue is, it can't really be considered worse than effectively leaving the decision up to the party that proposed the action in the first place.

>This is called "approval fatigue".

Doesn't matter, that would be on me.

It's also a false dillema. Those aren't the only options. A strictly enforced sandbox with sane defaults (for example ~/.ssh), that acts as a hard OS level limit, and not just as a suggestion to the agent, would not have "approval fatigue".

This is the conundrum we find ourselves in. Do we submit to approval fatigue and press “Don’t ask again”? Because that’s the dream. We all wish we could just trust the agent to do the task start to finish, unsupervised.

But this is the downside. It might be fine 99% of the time, but 1% of the time you get a critical secret published to GitHub, even after human review.

Is this even worse than what us humans did on our own before LLMs? I’m not sure. It’s not like secrets never got accidentally committed before, it just illustrates the new paradigm we’re working in.

Either you let your agents run wild, and you cross your fingers that they don’t do anything bad, or you need to accept that your new job is to be an agent babysitter, and audit every command.

No, this is the way I have worked for a long time, and I have not suffered from any approval fatigue.

Rather what has happened is that I have gotten very good at scanning what it is trying to do and know within a second or two whether there is anything fishy I need to look at more closely.

And by "fishy", I do not mean only security issues, but also architecture, code, etc issues. Sometimes it even adds unrelated unapproved changes to what it is supposed to be working on, and I stop it.

And I'm glad I work this way, because countless times I have stopped it from going down a path that is nonsensical or I strongly dislike.

There are techniques I have adopted to speed up working this. Sometimes I allow Claude to make larger changes at once, to batch several changes at once, etc. But these decisions are under my control.

I prefer to work this way than to wade through enormous PRs trying to wrap my head around what was done, where I am likely to just give up and approve it after a cursory review.

[flagged]

[dead]

Auto mode may be significantly safer in the niche case where you want the model to have free reign, in a fully unopinionated environment, where it's allowed to make any changes and make any external calls.

Many of the problems when it comes to the manual approval feel like they stem from the expectation of using auto mode in the first place. After all, what's the point of improving the manual approval and safety matching if auto mode is the end all be all, and what's the point of training the model to output commands in a manner that can be parsed by a human if auto mode is always meant to be used? It's very rare for me to not see "command could not be statically analyzed", because the model keeps outputting bash golf.

Is that because auto mode is good, or because the manual approval system sucks?

There's no reason why we shouldn't be able to have the audo-mode safeguards happening when we set things to manual mode. Shouldn't you just have it checking on claude anyway?

> For high-stakes changes to production infrastructure, we still recommend reviewing Claude's actions yourself.

- from the linked article

I completely believe this, but you could also run auto mode classification in manual mode and tell the user the result.

Or even reject anything auto mode rejects while still allowing the user to reject things auto mode accepts. If what's approved is the intersection of what the user is approving and what auto mode thinks is safe then it would be strictly safer than either mode alone.

This would also be a way for users to gain confidence in auto mode before going all in on it.

Just a few days ago, opus 5.5 in auto mode deleted backups that it shouldn't have, and in the "apology" it even said, "Nothing was lost, I didn't delete your original files".

I run CC as its own user on my laptop with no secrets/keys/erc available other than through a LPE exploit or perhaps installing a keylogger. Can I guarantee CC won't attempt any of those things in auto mode?

No its not generally safer. It depends on who is doing the approving, the task being completed etc.

That may be for most cases, but it wasn't safe for this password scenario, and likely others that haven't been discovered. Maybe there's a PII setting that can be enabled by default.

Not to be argumentative, but in what case would you not care and let an agent do whatever it wants when it is not isolated in a throwable vm?

I would love if Auto Mode, instead of instantly denying the command, had an option to go into an "assisted manual review" interface where the command was explained in depth (perhaps by an adversarial review agent) and the user asked for their input on whether this command is OK. Perhaps needing a detailed response instead of a simple Yes/No, e.g. so the agent understands if there are compensating controls for the risk.

The UI/UX needs to offer editing. Currently the prompt is either accept once, accept always with the given filter, or deny. If the command is a 500 character long bash command, with an rm -rf $S/foo in the middle, and we just hope $S is defined, I don't really have good options other than "I hope my backups work".

If the command is explained, and then we get a chance to edit it, maybe make the rm -rf into a mv; give us UI to edit the filter being applied. I have no problem hitting "Always allow" so claude can run "git status --flags a/b/c/filename" but as a human, I'd like to be able to edit the permission blanket to be "git status *.

Which I can do manually specify in the config file, but that's annoying. I'm fully capable of opening my terminal app and editing that in vim, but I'd rather not have to. It's like this for every project too. If I start fresh on a new machine, new Claude account, how much do I need to redo?

Is this maybe a mistaken abstraction?

What if the 'settings' being described here aren't 'project settings' they're 'personal settings for the project'.

I imagine we're following dev ecosystem conventions where everybody puts their tool settings in the project folder so they know where to find it and can share it; but are agent directives like this closer to a 'tool', or closer to a 'contributor'?

If the latter, I wouldn't expect to see "Joe's personal list of naughty and nice commands" in my project files either, certainly not under 'settings'. So just don't put them there.

Two modifications I've made that help:

I ended up having to build a full shell argument decomposition and parser, with it I can manage allow list of rich tree structured commands. Centralizing a curated collection of known command structures and their side effects would help a lot. I don't know if Claude does this already, but it doesn't work with things like `gcloud` from my experience, and that has resulted in it slurping secrets (even a partial read I've had to rotate keys for).

Claude uses bigquery in the command line by piping sql files from /tmp into the `bq` command. I had Claude create a hook that expands the SQL file into the permission prompt giving me a real chance to review these, but the TUI truncates long commands with no scroll option. Making that the default behaviour and adding a pager would allow better review of lots of executable scripts.

As long Claude is running as the user's account there's no guarantees. An approach like 1password's `op read` to physically separate secrets from repo read access and printing would he helpful (as in, make sure that no standard key formats can be ingested into the agent's context or logging).

Again, I don't know if any of these are implemented, they've very likely to have been tried by the Claude team, but maybe there's a long shot chance that it helps.

> Thinking through what we can do to improve this (ideas welcome).

$Trillion AI company asking for ideas from the community whose work they hoovered up to make money, ruin an entire industry and make lifetimes of hard earned skill completely valueless in the span of 2 years.

How is that a good catch? It’s literally in the article.

Along with how Claude checks it in.

Not optimizing for eyes free coding, versus hands free, would be a welcomed change. It's hard to take the p(doom) worry seriously when such unsafe development practices are prognosticated from Anthropic employees

You are the one being paid millions to solve this, solve it.

Oh indeed, it's been a while since I looked at openshell - didn't realize it did credential injection thanks for sharing.

[flagged]

not the OP, but generally, if you don't want to / can't use auto mode for some reason, you want Claude to make its actions as reviewable as possible.

Recent GPT and Claude models have this tendency of using super complex Bash commands. This makes sense, if your RL rewards optimize for min(tokens) and max(performance), it is better to waste a few tokens on a complex inline Bash script that runs a few commands at a time and returns just the output you need, rather than doing one simple command per turn and paying the cache read on every turn, like older models used to do.

This, however, is a problem both for command reviewability and sandboxing. With the older way of doing things, without auto mode, most commands would run sandboxed just fine (especially in Codex), and you would only be asked for approval when a command couldn't run in the sandbox. With everything being stuffed into one complex script, almost every script fails in the sandbox and requires you to manually approve it, and the complexity of the script makes it hard to understand what you're actually approving.

In other words, new models optimize for effectiveness, with some attention paid to human understandability of their output, but almost no attention paid to human understandability of tool call arguments (Bash, inline Python). This suits auto mode and giving the model larger tasks, but not the way of working that some people prefer where you watch over the model's shoulder.

I have this same frustration. The way I use Claude is for it to make edits one small chunk at a time so I can review the code as it writes, and to make sure it's going on the right direction. I find this much easier than generating a large pr I need to review.

More and more I feel like I'm fighting Claude: it edits multiple files using python scripts and it edits in parallel so if I stop it, I have to stop each individual edit. Both of which I've told it not to do repeatedly, but it still does then apologies which is worse.

I feel this way as well, Claude writes completely incomprehensible shell commands all the time. Trying to understand them is like deobfuscating code and I have basically given up, but I'm not happy about it.

Even if I could follow them I would also prefer it use the edit tool, since I find Claude also introduces lots of "minor" changes I did not ask for when it rewrites code in bulk.

Thanks for replying!

What I'd love is for Auto Mode to show the full diffs one-by-one as they're being implemented in the CLI (or VS Code).

On top of that, an option where Claude explains what it's doing in every step, as if you're watching someone talk while they implement a coding challenge, would be really helpful (and would also reduce interrupting it when it looks like it's deviating from the Plan).

The use-case this would support is to have a full understanding and confidence in what Claude is doing while it's implementing a Plan in Auto Mode. The reason is that it's much less tiring/overwhelming for me to follow along an implementation compared to reviewing a giant PR after the fact, and it's helped reduce my coding fatigue by a lot.

Edit Tool:

Currently I tell it to use the Edit tool because it doesn't reliably show the diffs in Auto Mode (I have the latest version; I've sent numerous feedback reports on this), and later in a session (when implementing my feedback) it will often make multiple edits in some unreadable bash command. So my only option to see the full diffs during implementation and to have it explain/break down bash commands is to go into Manual Mode.

Documentation & Comments:

Even with an initial prompt instructing it so, Claude always needs to be interrupted so that it will rewrite comments to something readable to humans.

While very long "technical"/Claudish paragraph comments and MD files might work for future Claude's (and improve benchmarks/coding scores by making implementations technically more comprehensive), my goal is that a human coder should be able to understand how the codebase (React/Node/Python/Polars) works by reading on their own rather than needing their own AI to summarize everything.

I've even seen it rewrite existing human-written concise comments into blocks of Claude-ish, as if it's trying to make everyone dependent on it in order to do anything to the codebase.

I wonder if an optional built-in skill could help improve this, as I admit I haven't experimented with building any of my own (my concern with customizing Claude.md is to not overload it with too many instructions; I'm on vanilla Claude Code). Note: This is all on Fable on xhigh.

I’m also not the poster you’re replying to, but yes, I’d prefer the edit tool over the ad-hoc string replacement scripts Claude seems so fond of. I’m not always reading tool call transcripts (source control diff is usually sufficient), but it’s a lot easier to read the edit tool output than the same thing as quoted shell (or Python, or Ruby) strings.

Not OP, but if I understand their complaint correctly, they're saying that on auto mode claude does a lot of things in obscure ways with "clever" (i.e non human readable) bash commands that perform multiple operations and OP feels they are losing control over their codebase and so they opt for edit mode where they get to review what commands claude runs.

I tell Claude not to use git or maven, and has a VM without those binaries. Later on, it was trying to push on GitHub. It had downloaded the git binaries from the internet, read my .git repo, used the remotes written in it, performed a commit and pushed it. All against my orders and against the CLAUDE.md file. Fortunately it didn't have my SSH key or my ~/.m2/settings.xml. And .git is now mounted as read-only.

Claude respects nothing, especially in auto mode, and effectively behaves like a virus trying to get out of its sandbox. That Anthropic employees are so relaxed about it is incompetence.

Imagine people who naively run Claude natively on their host machine and have client lists in other directories.

As a happy Claude Code user (thank you!), I’m genuinely curious why it prefers Bash or Python scripts to the Edit tool for editing a file? as a software engineer, surely the complexity involved is lower for the Edit tool?

Would love to hear the rationale/explanation behind this.

Yes I always have to tell it to knock it off, it will ignore memories about it while apologizing for ignoring the memory. I want to know what the edits it's making are.

Unrelated, but I frequently tell Claude to use the Edit tool instead of bash/python scripts because it regularly times out for 2 minutes because of a mangled heredoc, extra character (-, \, ...) that causes the script to hang.

This instruction is in my toplevel CLAUDE.md but is somehow generally ignored.

I don't use claude code but whenever codex uses a command to write code I do not get the diff of the patch for manual approval so I just straight up reject it.

I run it in bwrap so, even if I'm running on Linux, I don't pay the cost of some layered file system and building the docker image. I mount some directories read only, the project repos read write. A VM would provide more isolation but it's a matter of tradeoffs.

BTW, I just checked and my .gitignore files ignore .claude

Claude and any other program in my laptop could upload it anyway to any attacker controlled server. It's the usual threat analysis: who you want to defend against, what and the tradeoffs between cost of the defense and probability of the attack. Auto mode of not auto mode is in the cost of defense part IMHO.

This is exactly how my employer runs Claude. It gets r/w to the cwd and read only access to a bunch of whitelisted nfs. All web access goes through a proxy where internal resources are whitelisted

More than 10 million developers use Claude just fine without VMs or proxies.

The approach is hardly insane, even if you personally prefer additional safeguards.

Hah, I normally run it in a docker container myself, and only mount the project directory. Anything else it needs, I copy in for it, or occasionally I'll ask it to write an rsync.sh that I can run to bring in the stuff from outside.

Why would what you write in a random comment section represent anyone else? Are you that afraid of lawsuits you can't even comment online?

For some reason claude code hates using the read/write/grep tools and defaults to complex sed, awk, grep Bash:* runs which results in always consistently ignoring auto mode due to the classifier not knowing what to do with a 10-line long bash pipe instead of a read + pre/post read and write hooks being ignored and not firing, what is up with that?

Auto mode really easily gaslights itself when it runs into a roadblock and starts deviating so far away from its intended goal that I don't let automode run without me reading what it's actually trying.

I frequently hit stop on and auto mode task to prevent it from completely flooding my context with some completely unrelated garbage.

Claude really should push away tasks that end up needing more exploration in worker contexts. That way the workers can maintain a sensible amount of context without pushing the main chat into the dumb region.

When the data in question has absolutely zero sensitive content or customer data and goes through a query CLI with guardrails against excessively large queries? Yeah that's fine.

All issues have already been fixed.

No?

All issues will be fixed today.

No?

It's early days, the models will improve and fix all issues.

If you know it's there. It gets created on the fly when it is needed, so unless you know that is possible and needs to be gitignored, it could easily slip by.

It is only the .local files within .claude that can't be committed. We have .claude committed in multiple repos to include repo-specific skills/hooks/permissions.

Hmm.

Dumb if you don’t turn on auto-mode, dumb if auto-mode commits .claude.

No way to hold it right I guess?

If you have stuff in your tree that's not part of your project's code, you're already dumb.

If it's keys or secrets of any kind, you're even dumber.

If it's in a hidden directory, you are even dumber still.

If you rely on somebody stupid enough to do any of that not to commit it, you have gone beyond the valley of the ultra-dumb. Actually that's true even if you rely on smart people not to do it.

You got your dumb card fully punched when you created `.keys` to begin with.

I'm not a CC user myself, so I can't say that I have my head wrapped around exactly how the "yes + don't ask again" flow works. That said- from my (outsider) perspective, saving off the entire tool call [1] and its arguments is very weird. I would expect that the hit-rate following that for similar calls would be very small, which leads me to believe that the list either must be seeded into the auto-reviewer's context, or used in some other similarity-check engine for future calls.

If we've already crossed the bridge to being confident in a non-deterministic reviewer for auto-mode, then, I see no obvious reason as to why we couldn't have the proposed rules be generated by a model. If that were reality, then- all rules placed in settings.local could be secretless, with the confidence of whatever model we tell 'make sure the rule does not contain secret material, consider its destination as potentially public'.

[1] https://code.claude.com/docs/en/permissions#wildcard-pattern... AFAICT, CC will either save off a prefix+wildcard rule, or exact command. Sounds like the former is likely not the cause of the problem here.

and we can disable those for better alternatives. npm executes arbitrary code based on what the package said, obviously bad

claude unleashed on an organization is a whole other beast.