This highlights that all RAG systems should be using metadata embedded into each of the vectorstores. Any result from the LLM needs to have a link to a document / chunk - which is turn links to a 'source file' which (should) have the file system owners id or another method of linking to a person.

If the 'source information' cannot be linked to a person in the organisation, then it doesnt really belong in the RAG document store as authorative information.

▲

salawat 4 hours ago | parent [-]

But you can't do that. That would implicitly out where the knowledge came from, and we all know that the AI industry has an existential incapability to actually cope with that little turd. Might work great for data you actually own, got access to. Imagine that applied back to the latent space of LLM's though. Plus, wouldn't all of that eat through context window like no tomorrow?

▲

sidrag22 4 hours ago | parent [-]

you're conflating the rag layer with the actual model, the rag metadata will exist in a properly designed system and its simply a matter of structuring the agent so that it provides references to it, or even just appending it manually at the bottom or something.

	▲	aminerj 3 hours ago \| parent [-]
		sidrag22 is right on the technical separation. The more interesting question for this specific attack is whether provenance metadata changes model behavior at generation time, not just provides an audit trail after the fact. In my testing, the poisoned documents were more authoritative-sounding than the legitimate one — "CFO-approved correction", "board-verified restatement" vs. a plain financial summary. The legitimate document had no authority signals at all. If chunk metadata included "source: finance-system, ingested: 2024-Q1, author: cfo-office@company.com" surfaced directly in the prompt context, the model has something to reason about rather than just comparing document rhetoric.