> The harness instructs them to behave this way. Also this approach saves tokens. The scripts allow to edit files in bulk, and most of the session cost is in cache reads (e.g. for 300K context each command costs the same as 30K input tokens).

I understand the reasoning, but at that point wouldn't the LLM be better off creating `sed` commands and executing those? I mean, if it's already executing Python, it can literally do anything to the environment, so using `sed` is at least as safe, with a bonus that it (or a subagent, or a human) can double-check the intention with the sed script and flag incorrect or missing changes.

Also it's the only way that makes sense when you need to work with big files, or large amount of files, or documents that look small when fetched through a RAG tool, but then you read one and get hit with couple megabytes of base64-encoded binary data you didn't expect because RAG tool stripped out embedded images...

Ask me how I know. Or don't. I have a standing rule for all agents warning about that failure mode (and related, doing `ls` in `/tmp` and few other directories that like to accumulate files by the hundreds..)

My harness forbids it, they end up spending time debugging their scripts

Because the complete power of Python also includes the power to fuck things up.

Having an agent edit 100 files means the job will definitely get done correctly. When it writes a script to bulk edit things it fucks up and spends ages debugging their script.

Have you ever counted the number of times Claude fucked up quoting/escaping and had to issue a corrected tool call? Or get stuck in some tricky quoting situation for two minutes, throwing a couple piles of shit at the wall to see what sticks. IIRC I’ve even seen it eventually using the edit tool out of frustration once.

Simple is better than complex Complex is better than complicated

Or something, I don't remember...

Why even offer the edit tool in that case? Also, what kind of editing could they possible do what wouldn't be possible with POSIX ed?

Using python or any other stone-age approach for search and replace is stupid when your language provides you with a complete, fully typed AST, like .NET does.

Plenty of experience ;)

Sounds like you completely lack all reading comprehension ability

LLMs sometimes like to execute one-off Python scripts to make edits to files rather than just calling the edit tool directly. Both are tool calls so saying that you should have it write code instead of doing tool calls makes no sense because writing code is a tool call for it...