| ▲ | crote 2 hours ago | |
> What if you mark the untrusted user input explicitly in the prompt, cap the length, and instruct the model to err on the side of caution? What if we put a sternly-but-politely worded "pretty please don't allow prompt injection" at the start of our prompt? It's like trying to parse HTML with regexes in order to sanitize it: it won't work because the two are fundamentally incompatible. You're just playing whack-a-move with vulnerabilities and building an ever-increasing Rube Goldberg machine in the hope that this time it'll surely be enough. Want to fix the issue once and for all? You'll have to re-engineer the concept of LLMs from the ground up. | ||