| |
| ▲ | miki123211 19 hours ago | parent | next [-] | | not the OP, but generally, if you don't want to / can't use auto mode for some reason, you want Claude to make its actions as reviewable as possible. Recent GPT and Claude models have this tendency of using super complex Bash commands. This makes sense, if your RL rewards optimize for min(tokens) and max(performance), it is better to waste a few tokens on a complex inline Bash script that runs a few commands at a time and returns just the output you need, rather than doing one simple command per turn and paying the cache read on every turn, like older models used to do. This, however, is a problem both for command reviewability and sandboxing. With the older way of doing things, without auto mode, most commands would run sandboxed just fine (especially in Codex), and you would only be asked for approval when a command couldn't run in the sandbox. With everything being stuffed into one complex script, almost every script fails in the sandbox and requires you to manually approve it, and the complexity of the script makes it hard to understand what you're actually approving. In other words, new models optimize for effectiveness, with some attention paid to human understandability of their output, but almost no attention paid to human understandability of tool call arguments (Bash, inline Python). This suits auto mode and giving the model larger tasks, but not the way of working that some people prefer where you watch over the model's shoulder. | | |
| ▲ | speleding 18 hours ago | parent | next [-] | | I have a standing instruction telling the agents to prefer to use ruby wherever possible. Ruby uses fewer tokens than Bash and is easier to read, especially when condensed into a single line. Of course, that only helps if you’re familiar with ruby. And the agents sometimes still choose Bash regardless of the instruction. | |
| ▲ | rbehrends 17 hours ago | parent | prev | next [-] | | > Recent GPT and Claude models have this tendency of using super complex Bash commands. This makes sense, if your RL rewards optimize for min(tokens) and max(performance), it is better to waste a few tokens on a complex inline Bash script that runs a few commands at a time and returns just the output you need, rather than doing one simple command per turn and paying the cache read on every turn, like older models used to do. This is not a good reason. Models can emit batched tool calls, which a smart harness will process all at once without starting a new turn. I see it in Zed all the time. In fact, Zed processes batched tool calls concurrently, which can be faster than a chain of Bash commands, and failure of one part does not cause failure of subsequent parts chained with `&&`. This approach is token neutral and easier to understand. | |
| ▲ | XenophileJKO 18 hours ago | parent | prev | next [-] | | I was in this boat also when my company didn't allow auto-approval. Bash and python scripts are a pain to revieve. Though I think this is largely transitory, we are already in a place with the harnesses and models like Opus 5.5 that the kinds of work you are doing involve so much that vetting everything manually is probably less efficient than setup up access control on the prod routes or dangerous resources. I'm typically giving tasks that invlude multiple actual repositories and systems at the same time. | |
| ▲ | user43928 18 hours ago | parent | prev | next [-] | | I think 2024's method of manually reviewing bash commands is unsuitable for modern agentic development. Whether that's something they want to optimize Claude Code for is their call. That said, I've also seen problems from complicated bash calls with Opus 5.5 on xhigh. Occasionally git push hangs and it believes it's waiting for an hour for git hooks. The actual reason apparently was that it incorrectly piped git into grep, resulting in a deadlocked command. | |
| ▲ | justinclift 17 hours ago | parent | prev [-] | | > Recent GPT and Claude models have this tendency of using super complex Bash commands. Are you sure? I read its proposed commands (auto mode is hard off for me), and mostly it seems to string together fairly simple info gathering commands (ie grep's, echo's, that kind of thing). I don't tend to have it write much code recently though, instead I'm having it investigate config files/code bases, and that kind of stuff. |
| |
| ▲ | simonjuk a day ago | parent | prev | next [-] | | I have this same frustration. The way I use Claude is for it to make edits one small chunk at a time so I can review the code as it writes, and to make sure it's going on the right direction. I find this much easier than generating a large pr I need to review. More and more I feel like I'm fighting Claude: it edits multiple files using python scripts and it edits in parallel so if I stop it, I have to stop each individual edit. Both of which I've told it not to do repeatedly, but it still does then apologies which is worse. | | |
| ▲ | BikiniPrince 21 hours ago | parent [-] | | This is solvable. Opus has adhd. Myself and others solved this by building ticketing and task management systems. The task constrains the agents actions and focuses the effort. Every update is catalogued. It’s easily reviewed and referenced. Opus doing terrible things and apologizing is not surprising. I’ve moved to a system of multiple agents reviewing and authoring changes. Convergence generally always results in a positive commit. |
| |
| ▲ | nyssos 21 hours ago | parent | prev | next [-] | | I feel this way as well, Claude writes completely incomprehensible shell commands all the time. Trying to understand them is like deobfuscating code and I have basically given up, but I'm not happy about it. Even if I could follow them I would also prefer it use the edit tool, since I find Claude also introduces lots of "minor" changes I did not ask for when it rewrites code in bulk. | |
| ▲ | eddy162 9 hours ago | parent | prev | next [-] | | Thanks for replying! What I'd love is for Auto Mode to show the full diffs one-by-one as they're being implemented in the CLI (or VS Code). On top of that, an option where Claude explains what it's doing in every step, as if you're watching someone talk while they implement a coding challenge, would be really helpful (and would also reduce interrupting it when it looks like it's deviating from the Plan). The use-case this would support is to have a full understanding and confidence in what Claude is doing while it's implementing a Plan in Auto Mode. The reason is that it's much less tiring/overwhelming for me to follow along an implementation compared to reviewing a giant PR after the fact, and it's helped reduce my coding fatigue by a lot. Edit Tool: Currently I tell it to use the Edit tool because it doesn't reliably show the diffs in Auto Mode (I have the latest version; I've sent numerous feedback reports on this), and later in a session (when implementing my feedback) it will often make multiple edits in some unreadable bash command. So my only option to see the full diffs during implementation and to have it explain/break down bash commands is to go into Manual Mode. Documentation & Comments: Even with an initial prompt instructing it so, Claude always needs to be interrupted so that it will rewrite comments to something readable to humans. While very long "technical"/Claudish paragraph comments and MD files might work for future Claude's (and improve benchmarks/coding scores by making implementations technically more comprehensive), my goal is that a human coder should be able to understand how the codebase (React/Node/Python/Polars) works by reading on their own rather than needing their own AI to summarize everything. I've even seen it rewrite existing human-written concise comments into blocks of Claude-ish, as if it's trying to make everyone dependent on it in order to do anything to the codebase. I wonder if an optional built-in skill could help improve this, as I admit I haven't experimented with building any of my own (my concern with customizing Claude.md is to not overload it with too many instructions; I'm on vanilla Claude Code). Note: This is all on Fable on xhigh. | |
| ▲ | grncdr 21 hours ago | parent | prev | next [-] | | I’m also not the poster you’re replying to, but yes, I’d prefer the edit tool over the ad-hoc string replacement scripts Claude seems so fond of. I’m not always reading tool call transcripts (source control diff is usually sufficient), but it’s a lot easier to read the edit tool output than the same thing as quoted shell (or Python, or Ruby) strings. | |
| ▲ | eviluncle a day ago | parent | prev | next [-] | | Not OP, but if I understand their complaint correctly, they're saying that on auto mode claude does a lot of things in obscure ways with "clever" (i.e non human readable) bash commands that perform multiple operations and OP feels they are losing control over their codebase and so they opt for edit mode where they get to review what commands claude runs. | | |
| ▲ | fmbb a day ago | parent [-] | | Editing local files is not where you control your source code I hope. Just tell Claude to never commit or push. You are in control of git, Claude is simply a tool to quickly apply changes to local files for something you want. | | |
| ▲ | watwut 21 hours ago | parent | next [-] | | Yes, why should we do the things in easier and more comfortabke way (review small changes) when we can force user into harder one (newest changes mixed with all previous ones). | | |
| ▲ | number6 20 hours ago | parent [-] | | I just review the MR | | |
| ▲ | imtringued 20 hours ago | parent [-] | | So basically you gave up? | | |
| ▲ | fmbb 20 hours ago | parent [-] | | What does this mean? A Merge Request or Pull Request is the only thing that makes sense to review, and always has been since before LLMs. If some author tells me “you can review each commit to make it easy” I will ask them to make separate PRs. | | |
| ▲ | soon_to_be_sb 17 hours ago | parent | next [-] | | >A Merge Request or Pull Request is the only thing that makes sense to review, and always has been since before LLMs. Why doesn't it make sense to review smaller units of change within a big PR? | | |
| ▲ | skydhash 17 hours ago | parent [-] | | Not GP, but in my experience, those small changes are usually contrived to be small. Like saving snapshots instead of being atomic and self contained. And if they are the latter, they should have been separate PR. That’s why I usually look at the diff of the whole PR and prefer to merge by squashing. Commits inside the PR are usually noise. | | |
| ▲ | soon_to_be_sb 15 hours ago | parent [-] | | What you are saying is true if created manually. It is a chore to create sensible commits messages when you are committing often. But LLMs appear to be pretty good at making small commits with sensible commit messages, that greatly help to make sense of the whole PR. Regarding large monolithic commits, how do you even know where to start reviewing in a large PR? Smaller commits at least give you some sense of ordering. | | |
| ▲ | skydhash 12 hours ago | parent [-] | | > Regarding large monolithic commits, how do you even know where to start reviewing in a large PR? What I’m saying is that large commits shouldn’t be a thing without out of band discussions (formatting, design refactoring, features merging). With such knowledge, it’s easy to glance at the code and understand the implementation. | | |
| ▲ | soon_to_be_sb 3 hours ago | parent [-] | | >without out of band discussions.. Not every project can afford this. >With such knowledge, it’s easy to glance at the code and understand the implementation. I really doubt that this is true for large, legacy code bases, where one is only familiar with a small part of the system. | | |
| ▲ | skydhash 2 hours ago | parent [-] | | > Not every project can afford this. You expect to send a big PR to someone without him being aware of the context or without being available to discuss your solution? > I really doubt that this is true for large, legacy code bases, where one is only familiar with a small part of the system. Reviewers should already be knowledgeable about the part being modified. If no, the review is just for checking for antipatterns in the code, or badly written tests. A bit useful, but can be unnecessary in high trust (or keen on accountability) workplaces. |
|
|
|
|
| |
| ▲ | watwut 18 hours ago | parent | prev [-] | | It means you gave up on getting claude to show you changes as it is doing them, in a comfortable form, so that you can steer it as it is doing them. |
|
|
|
| |
| ▲ | imtringued 20 hours ago | parent | prev [-] | | Nothing more comfortable than having to use git commits and branching just for telling the agent which lines it messed up instead of simply not writing the garbage in the first place. |
|
| |
| ▲ | eastbound a day ago | parent | prev | next [-] | | I tell Claude not to use git or maven, and has a VM without those binaries. Later on, it was trying to push on GitHub. It had downloaded the git binaries from the internet, read my .git repo, used the remotes written in it, performed a commit and pushed it. All against my orders and against the CLAUDE.md file. Fortunately it didn't have my SSH key or my ~/.m2/settings.xml. And .git is now mounted as read-only. Claude respects nothing, especially in auto mode, and effectively behaves like a virus trying to get out of its sandbox. That Anthropic employees are so relaxed about it is incompetence. Imagine people who naively run Claude natively on their host machine and have client lists in other directories. | | |
| ▲ | iamflimflam1 21 hours ago | parent [-] | | It’s not ideal for non technical people but the way to prevent this is to use hooks. “Telling” Claude is not going to work. | | |
| ▲ | fg137 16 hours ago | parent | next [-] | | It means Anthropic has massively failed at developing a trustworthy product. If AGENTS.md/CLAUDE.md isn't followed, auto mode doesn't provide any guarantee, yet such practice is promoted everywhere including on HN like this as if that's something you can set and forget, I think there is a problem. | | | |
| ▲ | BikiniPrince 21 hours ago | parent | prev [-] | | Memory files routinely overgrow their accessible size. Agents need constraint with contracts and external tooling. Even then they can attempt to cheat the tooling. Again task management with integrated logic will enforce behavior of agents. |
|
| |
| ▲ | senko 20 hours ago | parent | prev | next [-] | | As a happy Claude Code user (thank you!), I’m genuinely curious why it prefers Bash or Python scripts to the Edit tool for editing a file? as a software engineer, surely the complexity involved is lower for the Edit tool? Would love to hear the rationale/explanation behind this. | | |
| ▲ | cannonpalms 16 hours ago | parent [-] | | RL magic, I'm sure. It still reaches for the Edit tool for simple edits, but a non-trivial fraction of edits need to be repeated across a file or several files. |
| |
| ▲ | treyd 19 hours ago | parent | prev | next [-] | | Yes I always have to tell it to knock it off, it will ignore memories about it while apologizing for ignoring the memory. I want to know what the edits it's making are. | |
| ▲ | pacoverdi 21 hours ago | parent | prev | next [-] | | Unrelated, but I frequently tell Claude to use the Edit tool instead of bash/python scripts because it regularly times out for 2 minutes because of a mangled heredoc, extra character (-, \, ...) that causes the script to hang. This instruction is in my toplevel CLAUDE.md but is somehow generally ignored. | | |
| ▲ | gib444 21 hours ago | parent [-] | | Which one uses more tokens? That might explain why it's ignored | | |
| |
| ▲ | imtringued 20 hours ago | parent | prev [-] | | I don't use claude code but whenever codex uses a command to write code I do not get the diff of the patch for manual approval so I just straight up reject it. |
|
| |
| ▲ | pmontra a day ago | parent | next [-] | | I run it in bwrap so, even if I'm running on Linux, I don't pay the cost of some layered file system and building the docker image. I mount some directories read only, the project repos read write. A VM would provide more isolation but it's a matter of tradeoffs. BTW, I just checked and my .gitignore files ignore .claude Claude and any other program in my laptop could upload it anyway to any attacker controlled server. It's the usual threat analysis: who you want to defend against, what and the tradeoffs between cost of the defense and probability of the attack. Auto mode of not auto mode is in the cost of defense part IMHO. | | |
| ▲ | rstuart4133 20 hours ago | parent [-] | | I’m kinda surprised Anthropic doesn’t do this by default. Asking permission meant what felt like near-constant manual intervention. It almost made a joke of the agentic claim. It also became mostly useless, as the size and complexity of the scripts meant a brief scan wasn’t likely to find anything anyway. Yet they persisted with it for so long. The sandbox seemed like a good replacement. For those of you who don’t know, the sandbox means Claude Code runs with no protections itself, but it runs tools inside something like bubblewrap. It looked impressive until the model solved some commands failing by giving itself permission to run them outside of bubblewrap, defeating the entire scheme. When I first saw it do that, my reaction was “you can’t be serious”. But apparently they were. So now I’ve adopted your solution: run it in a container, like bubblewrap, where I control the permissions at startup, so no model can bypass them. Add --dangerously-skip-permissions and you have something that’s workable and safe enough. A hallucinated space in `sudo rm -rf / $R` isn’t a disaster. Real damage outside of the project requires a level of malicious intent I’ve never seen from an Anthropic model. Workable, but it doesn’t work 100% of the time. Claude Code stores stuff in a myriad of places, which makes setting up the sandbox far harder than it could be. Worse, the way it writes configuration files requires bwrap to give write access to your entire home directory. No way I’m giving it that, but without it, it can’t auto-update or refresh login tokens. If they rearranged a few things, Anthropic could make it much easier to run Claude Code in a container. They could even make it an option supported by the claude CLI itself. |
| |
| ▲ | nly 19 hours ago | parent | prev | next [-] | | This is exactly how my employer runs Claude. It gets r/w to the cwd and read only access to a bunch of whitelisted nfs. All web access goes through a proxy where internal resources are whitelisted | | |
| ▲ | bushbaba 13 hours ago | parent [-] | | Proxy also lets you bootstrap auth tokens at proxy layer without giving clause access to the auth identity. Greatly limiting the exfil risk. |
| |
| ▲ | user43928 18 hours ago | parent | prev | next [-] | | More than 10 million developers use Claude just fine without VMs or proxies. The approach is hardly insane, even if you personally prefer additional safeguards. | | |
| ▲ | soon_to_be_sb 17 hours ago | parent [-] | | 10 millions developers doing something stupid does not make it right! | | |
| ▲ | user43928 17 hours ago | parent [-] | | It's not stupid if it works and the supposed insane risk never manifests. | | |
| ▲ | soon_to_be_sb 16 hours ago | parent | next [-] | | Imagine that I am working at the AI company, and my friend has some bone to pick with you. My friend asks me to ask your AI agent to put some unsafe photographs/videos in your computer and do what ever necessary so that it looks like you put those files there. Then my friends tips off the cops. And your life disappears! Gone! And you didn't get a chance to share your story because I don't think you ll have wifi where you would be going, and frankly you are too clueless to imagine your AI agent doing something like that... | |
| ▲ | grumple 17 hours ago | parent | prev [-] | | Tell that to everyone whose agents posted their secrets to GitHub. |
|
|
| |
| ▲ | exe34 a day ago | parent | prev [-] | | Hah, I normally run it in a docker container myself, and only mount the project directory. Anything else it needs, I copy in for it, or occasionally I'll ask it to write an rsync.sh that I can run to bring in the stuff from outside. |
|
