From the perspective of the software vendor, it may be a semi-regular occurrence that they learn that users are being actively harmed by a software vulnerability exploited in-the-wild. So that's an argument that developers have a moral obligation to maintain the ability to push updates their users without delay.

Waiting for the user to click "Check for updates..." is effectively pushing this responsibility onto the users, the vast majority of whom lack the information and expertise needed to make an informed choice about the risk.

We're talking about Claude Code, the frontend to the online, hosted LLM inference suite, right? The auto-updater isn't where they get their usage metrics.

Simply running the software means trusting the developer. But even then, do you really read the commits comprising the latest Firefox update? How would I review the updates for my cell phone? I just hit "okay", or simply set up auto updates.

There's a term "eggcorns" for these logical misinterpretations - https://en.wikipedia.org/wiki/Eggcorn

Not all programs misbehave in this way. Signal desktop lets you turn off this vulnerability, and of course iOS apps and normal macOS apps are not allowed to self-modify, as it breaks their signature.

https://github.com/signalapp/Signal-Desktop/issues/4578

https://github.com/syncthing/syncthing-macos/issues/122

I would say it's neither, it's complacent misuse by the user. As you allude to we generally already are, but non-deterministic & especially 'agentic' AI makes the stakes/likelihood of it going wrong so much higher.

Don't use an MCP server with permission (capability) to do more than you want, regardless of whether you think you're instructing the AI tool do the bad thing it's technically capable of.

Don't run AI tools with filesystem access outside of something like a container with only a specific whitelist of directory mounts.

Assume that the worst that could happen with the capability given will happen.

> Terminal and Bash or any shell can do this, if the user sucks.

But at least they will do it deterministically.

There are scenarios in which you allow it to run python or uv for the session (perhaps because you want it to run tests on its own), and then for whatever reason it could run `subprocess.run("rm -rf / --no-preserve-root".split())` or something like that.

I use it in a container, so at worst it can delete my repository.

By default it asks before running commands. The options when it asks are something like

[1] Yes

[2] Yes, and allow this specific command for the rest of this session

[3] No

One easy way to accidentally give Claude permission to do almost anything is to tell it that it’s allowed to run “find” without confirmation.

Claude is constantly searching through your files and approving every find command is annoying.

The problem is, find has a --exec flag that lets it run arbitrary bash commands. So now Claude can basically do anything it wants.

I have really been enjoying Claude in a container in yolo mode though. Seems like the main risk I am taking is data exfiltration since it will has unfettered access to the internet.