| ▲ | bandrami a day ago | |||||||
Maybe I'm missing something but does this idea need a "theory"? There's zero sideband here; everything is just context. "Injection" is just kind of baked in to the design. | ||||||||
| ▲ | geoffschmidt a day ago | parent | next [-] | |||||||
I think their work earns "theory" because it makes specific predictions both about how to make more effective prompt injection attacks and what activations you'd observe in the LLM during those attacks, and can also be plausibly extrapolated to suggest useful future research directions. | ||||||||
| ▲ | yunwal a day ago | parent | prev | next [-] | |||||||
At this point I think it's similar to reporting a particularly effective social engineering practice. It's not particularly surprising that it works or that it exists, but it's still noteworthy. | ||||||||
| ||||||||
| ▲ | zby a day ago | parent | prev | next [-] | |||||||
They do predict what injections might be effective - so it is a theory. I don't know how novel it is and it is not very deep (as you noted the general mechanism is quite obvious) - but they do it quite systematically so it is useful. | ||||||||
| ▲ | jackb4040 a day ago | parent | prev [-] | |||||||
I was gonna say, anyone who's copy-pasted one LLM conversation into another already intuitively understands all this. | ||||||||