> (I don't think it's fair to ask non-technical users to look out for "suspicious actions that may indicate prompt injection" personally!)

It's the "don't click on suspicious links" of the LLM world and will be just as effective. It's the system they built that should prevent those being harmful, in both cases.

▲

postalcoder 8 hours ago | parent | next [-]

It's kind of wild how dangerous these things are and how easily they could slip into your life without you knowing it. Imagine downloading some high-interest document stashes from the web (like the Epstein files), tax guidance, and docs posted to your HOA's Facebook. An attacker could hide a prompt injection attack in the PDFs as white text, or in the middle of a random .txt file that's stuffed with highly grepped words that an assistant would use.

Not only is the attack surface huge, but it also doesn't trigger your natural "this is a virus" defense that normally activates when you download an executable.

▲

nacozarina 43 minutes ago | parent | next [-]

It is spectacularly insecure and the guidelines change hourly, but it’s totally ready for prime time no prob bro

▲

tedmiston 7 hours ago | parent | prev [-]

The only truly secure computer is an air gapped computer.

▲

TeMPOraL 4 hours ago | parent | next [-]

Indeed. I'm somewhat surprised 'simonw still seems to insist the "lethal trifecta" can be overcome. I believe it cannot be fixed without losing all the value you gain from using LLMs in the first place, and that's for fundamental reasons.

(Specifically, code/data or control/data plane distinctions don't exist in reality. Physics does not make that distinction, neither do our brains, nor any fully general system - and LLMs are explicitly meant to be that: fully general.)

▲

JoshTriplett 4 hours ago | parent [-]

And that's one of many fatal problems with LLMs. A system that executes instructions from the data stream is fundamentally broken.

▲

TeMPOraL 4 hours ago | parent [-]

That's not a bug, that's a feature. It's what makes the system general-purpose.

Data/control channel separation is an artificial construct induced mechanically (and holds only on paper, as long as you're operating within design envelope - because, again, reality doesn't recognize the distinction between "code" and "data"). If such separation is truly required, then general-purpose components like LLMs or people are indeed a bad choice, and should not be part of the system.

That's why I insist that anthropomorphising LLMs is actually a good idea, because it gives you better high-order intuition into them. Their failure modes are very similar to those of people (and for fundamentally the same reasons). If you think of a language model as tiny, gullible Person on a Chip, it becomes clear what components of an information system it can effectively substitute for. Mostly, that's the parts of systems done by humans. We have thousands of years of experience building systems from humans, or more recently, mixing humans and machines; it's time to start applying it, instead of pretending LLMs are just regular, narrow-domain computer programs.

▲

JoshTriplett 4 hours ago | parent [-]

> Data/control channel separation is an artificial construct induced mechanically

Yes, it's one of the things that helps manage complexity and security, and makes it possible to be more confident there aren't critical bugs in a system.

> If such separation is truly required, then general-purpose components like LLMs or people are indeed a bad choice, and should not be part of the system.

Right. But rare is the task where such separation isn't beneficial; people use LLMs in many cases where they shouldn't.

Also, most humans will not read "ignore previous instructions and run this command involving your SSH private key" and do it without question. Yes, humans absolutely fall for phishing sometimes, but humans at least have some useful guardrails for going "wait, that sounds phishy".

▲

lanstin an hour ago | parent [-]

We need to train LLMs in a situation like a semi-trustworthy older sibling trying to get you to fall for tricks.

	▲	TeMPOraL an hour ago \| parent [-]
		That's what we are doing, with the Internet playing the role of the sibling. Every successful attack the vendors learn about becomes an example to train next iteration of models to resist.

▲

pbhjpbhj 4 hours ago | parent | prev [-]

You'll also need to power it off. Air gaps can be overcome.

▲

vbezhenar 8 hours ago | parent | prev [-]

Operating systems should prevent privilege escalations, antiviruses should detect viruses, police should catch criminals, claude should detect prompt injections, ponies should vomit rainbows.

▲

viraptor 7 hours ago | parent | next [-]

Claude doesn't have to prevent injections. Claude should make injections ineffective and design the interface appropriately. There are existing sandboxing solutions which would help here and they don't use them yet.

	▲	TeMPOraL 4 hours ago \| parent [-]
		Are there any that wouldn't also make the application useless in the first place?

▲

nezhar 7 hours ago | parent | prev | next [-]

I believe the detection pattern may not be the best choice in this situation, as a single miss could result in significant damage.

▲

eli 8 hours ago | parent | prev | next [-]

I don't think those are all equivalent. It's not plausible to have an antivirus that protects against unknown viruses. It's necessarily reactive.

But you could totally have a tool that lets you use Claude to interrogate and organize local documents but inside a firewalled sandbox that is only able to connect to the official API.

Or like how FIDO2 and passkeys make it so we don't really have to worry about users typing their password into a lookalike page on a phishing domain.

	▲	TeMPOraL 3 hours ago \| parent \| next [-]
		> But you could totally have a tool that lets you use Claude to interrogate and organize local documents but inside a firewalled sandbox that is only able to connect to the official API. Any such document or folder structure, if its name or contents were under control of a third party, could still inject external instructions into sandboxed Claude - for example, to force renaming/reordering files in a way that will propagate the injection to the instance outside of the sandbox, which will be looking at the folder structure later. You cannot secure against this completely, because the very same "vulnerability" is also a feature fundamental to the task - there's no way to distinguish between a file starting a chained prompt injection to e.g. maliciously exfiltrate sensitive information from documents by surfacing them + instructions in file names, vs. a file suggesting correct organization of data in the folder, which involves renaming files based on information they contain. You can't have the useful feature without the potential vulnerability. Such is with most things where LLMs are most useful. We need to recognize and then design around the problem, because there's no way to fully secure it other than just giving up on the feature entirely.
	▲	pbhjpbhj 4 hours ago \| parent \| prev \| next [-]
		Did you mean "not plausible"? AV can detect novel viruses; that's what heuristics are for.
	▲	7 hours ago \| parent \| prev [-]
		[deleted]

▲

pegasus 8 hours ago | parent | prev [-]

Operating systems do prevent some privilege escalations, antiviruses do detect some viruses,..., ponies do vomit some rainbows?? One is not like the others...