I mean ideally AI would be resilient to junk, don't you think?

Humans are pretty vulnerable to junk so I’m not sure.

Ideally, you'd run your own instance of this, I think.

I can see a product where you purchase a model that has basic training, and then, using the features outlined in the paper, it learns on the fly from your usage.

I can also see there being a secondary market for specially trained models, long-term memory filled with some specific skill, done in some specific way. To make a silly example, imagine buying a licence to Torvald's OS coding assistant, ready to insult your prs before you even commit them!(And possibly help you write code in Torvald's style too)

This would of course require Linus to use the model enough for it to learn,I won't comment on the likelihood of that happening: it's just a silly example after all