> This feels like it could be solved with a list of permissions that the user has to turn on when using 3rd party AI.

The device won't be able to ask for significantly more permissions than Apple asks for their own model for regulatory reasons, nor will it be able to convey the seriousness of granting the permission (e.g. immediately give unrestricted access to the vast majority of personal information/documents stored on the device).

But Apple also architected their system to justify not having constant permission prompts for access to sensitive data. And for regulatory reasons they also can't mandate that competing models have the same architecture.

The regulators and Apple (along with hopefully other AI companies) will need to work together to determine longer-term stable path forward.

Apple could have the same kind of permission dialogues with their own models (and they actually should). Each and every (first-time) use of a feature should:

  1) ask for permission explaining the scope
  2) warn you about the dangers with a confirmation / nevermind option

  1) Acme AI requires access to your email provider in order to execute this request. Grant / Deny
  2) You're about to let Acme AI read and send emails on your behalf, this might be dangerous due to X and Y. Do you want to continue? / Nevermind.

  1) Asks for access to a service
  2) Asks for a specific use-case of the service

2 is permission to act, but you might want to deny access to some parts such as sending email and scope to summarization