> you can imagine how a locally run LLM that was just part of the SDK/API developers could leverage would lead to better UI/UX

It’s already there for Apple developers: https://developer.apple.com/documentation/foundationmodels

I saw some presentations about it last year. It’s extremely easy to use.