Modern AI is built on data. Trusting that our conversations will not be used for training the model is a bit like giving a glutton their favourite food but making them promise not to eat it. Sure, why not.

I still expect that our conversations will not leave the premises (ie end up on the internet), because that would be something else, but other than that, I knew what I signed up for.

Don't normalize this. There are contractual obligations that we have to enforce in order to keep our privacy and humanity.