Remix.run Logo
roywiggins 5 days ago

Best you can do is constrain responses to follow a schema, but if that schema has any free text you can still poison the context, surely? Like if I instruct an agent to read an email and take an appropriate action, and the email has a prompt injection that tells it to take a bad action instead of a good action, I am not sure how structured generation helps mitigate the issue at all.