Thats not what they are asking. This paper is discussing documents in the training dataset poisoning the LLM for malicious behavior. This person are asking if anyone has deliberately put something in a private chat (presumably with retrain on my data turned off), to see if they can get it to leak across sessions from distinct users. I am positive this happens but I have not seen the proof. I also want to know the answer to this question.

Here are potentially relevant documents?

https://medium.com/secludy/fine-tuning-llm-on-sensitive-data...

https://spylab.ai/blog/non-adversarial-reproduction/

https://arxiv.org/abs/2601.18834

* with train on my data turned ON, yes. Though OFF would of course be even more notable!

Thank you – the non-adversarial reproduction paper ( https://arxiv.org/abs/2411.10242 ) nails it – from chat, to training corpus, to subsequent model. Though in my hasty read, it is not entirely clear whether the snippets it finds are nonces, i.e. present exactly once in the internet.