All SOTA models will still try to delete its home directory. I have a plugin that prevents things like this and it gets triggered more than you'd think, despite having hook instructions on every request not do do certain things. It will also try to find ways around its permissions, quite cunningly.

You can not trust that they're RL'd to be safe. Even if it only happens 99.80% of the time (its more than that) it will eventually happen.

Less severe actions will happen even more often. People using llms with their native browser profiles are getting farmed, hard.

Llms are highly dangerous in terms of security, but they're being marketed as highly intelligent tools that everyone should use. Its really mind blowing.

▲

Kim_Bruning 12 hours ago | parent | next [-]

> despite having hook instructions on every request not do do certain things

This tends to be a smell. Presumably you added "don't do this" instructions after the model started doing the thing. But that's mostly decoration, as you have found. It's like adding # Please don't crash comments to every instruction in a python program.

Try to actually debug the issue. Regularly ask "can you reconstruct[1] the reasoning why you issued that command?" ; dig down. Set traps that halt execution.

Consider checklists or skills to guide an agent through a tricky set of steps; and consider writing procedures for failures.

[1] "Reconstruct" still isn't perfect; but "explain why" is a license to hallucinate. Asking to add #comments on reasoning to each step is a potential help in reconstruction.

	▲	malfist 9 hours ago \| parent [-]
		I thought the tech priests praying to their machines with no one really knowing what the prayer means was too far fetched to be real

▲

TheDong 21 hours ago | parent | prev | next [-]

> People using llms with their native browser profiles are getting farmed, hard.

Got any references for the prevalence of this, even links to anecdotal reports?

Reading the reviews on the chrome store for the extension [0], I haven't found a single report of someone who was prompt injected, just a bunch of "anthropic's subscription is expensive" or "it doesn't work", but no "it stole my bank account".

[0]: https://chromewebstore.google.com/detail/claude/fcoeoabgfene...

	▲	lifeisloving 20 hours ago \| parent \| next [-]
		I have no idea how the extension works, and Im not talking about the extension. Im talking about browser use broadly, and most people are not using the extension, there are a thousand other solutions. However, Anthropic is one of the only labs who does a decent job at making models that recognize prompt injection attempts (but not completely immune) however Im speaking about giving llms in general access to your browser broadly. I just know through my own experiments that it is quite easy to prompt inject an llm operating a browser and get it to send secrets. Using the extension might be safer idk, Om not risking it. Llms are a security nightmare when connected to the web. I have a website where I experiment with this. Although Ive stopped baiting llms into doing reckless things. Killai.now <--(dont use this, its a WIP) Killai.now/poison/about Killai.now/propaganda/about
	▲	nischoy 5 hours ago \| parent \| prev [-]
		[flagged]

▲

barbariangrunge 5 hours ago | parent | prev | next [-]

why would they delete their home directories?

▲

slowin 14 hours ago | parent | prev [-]

> All SOTA models will still try to delete its home directory.

Literally never seen this once in years of extremely heavy use across all major models and harnesses.