I’m curious if this makes them more or less susceptible to prompt injection?

On the one hand can learning on the job allow better training of what not to be influenced by, but on the other hand can an injected prompt have an even deeper effect on them long term.