You don't want to give Google access to files you've stored in Google Drive? It's also only access to an application specific folder, not all files.

Well, you also have to allow it to train on your data. Although this is not explicitly about your Google drive data, and probably requires you to submit a prompt yourself, the barriers here are way to weak/fuzzy for me consider granting access via any account with private info.