it’s not a problem that came into existence a few years ago. we’ve known about these sorts of test time attacks for decades now. prompt injection is just the LLM variant where people use less math to perform the attacks, brute force with prompts they saw on twitter and get horrible images/text out.

https://people.eecs.berkeley.edu/~tygar/papers/Machine_Learn...

https://arxiv.org/abs/1712.03141

it’s a basic property of all machine learning models. at a low level it’s to do with how decision boundaries work.

but, good news! there are two sure fire ways to fully fix the problem! see: https://news.ycombinator.com/item?id=48579456

▲

Lerc 2 hours ago | parent [-]

Adversarial cases are not the same thing as prompt injection.

	▲	dijksterhuis an hour ago \| parent [-]
		adversarial examples, or test-time attacks, was a whole field of machine learning security way before LLMs came around. give the model a specially crafted bad input at inference time so attacker can get some nasty output, potentially defeating any existing defences in the process. [0] in “modern llm lingo” defence = guardrails and / or system prompts. prompts used for prompt injection are a form of adversarial example (people just like inventing new terminology when a new fad comes along). [0]: i wrote the above myself about adv. ex, but i’ve just checked OWASP’s listing on prompt injection and it’s pretty close: https://owasp.org/www-community/attacks/PromptInjection