We need to train LLMs in a situation like a semi-trustworthy older sibling trying to get you to fall for tricks.

That's what we are doing, with the Internet playing the role of the sibling. Every successful attack the vendors learn about becomes an example to train next iteration of models to resist.