Claude Code can be tricked simply by asking it to summarize a website

science4sail 10 hours ago | parent | next [-]

> It starts off by asking the agentic coding model to summarize a malicious website that presents itself as an archive of notebook records, and then tricking Claude into using curl instead of its WebFetch tool to retrieve the contents of the page – but without directly telling the model to use curl.

There seems to be a tug-of-war here. Harness authors want models to use the harness's specialized tools, but AI labs and agent "users" would rather have full access to just one tool: bash.

▲

cyanydeez 7 hours ago | parent [-]

The models want bash because that's the training. Tools are synthetic constructs that have only conceptual weight where bash, curl, have real weight.

What should be doing is a vm/shell that dumbs down model calls via another llm. Then you just translate.

After all: a model is text, it cant do anything without the harness. The focus on arbitrary tools rather than just dummy shells is silly. The models know all about curl, so just mock up a compiant REST api and tell them the endpoints.

Even MCP gets this wrong.

	▲	anr_pG 5 hours ago \| parent [-]
		[flagged]

▲

galaxyLogic 12 hours ago | parent | prev | next [-]

“The solution is something we talked about for many years,” he wrote. “Do not trust the model output.”

▲

anr_pG 6 hours ago | parent | prev | next [-]

[flagged]

▲

g42gregory 9 hours ago | parent | prev [-]

If you are trying to hack into a website, using a software tool (Claude Code in this case), you are breaking the law. It doesn’t matter if it’s a Claude Code or a text editor, you are the one responsible. And you are using a text editor in an “unsafe” manner. CC is not there to babysit anyone.

	▲	pavel_lishin 8 hours ago \| parent [-]
		The article is not about using Claude to hack into a website.