This seems more of a toy. I hope we get to eventually able to use a local Apple LLM model with more flexibility.

Something quite cool they shipped with the 26 family of operating systems is a new system-level API to let developers access the built-in on-device models directly: https://developer.apple.com/documentation/FoundationModels