I’m building a on-device AI App that supports a good variety of open-weight LLM Models. It’s mostly so that I can gain an intuition on what goes on when we do inferencing, but my longer term plan is

1. Support on-device RAG to allow chatting with your own documents on mobile offline 2. Support MCP on-device, taking advantage of information that’s (only) available on your phone, like calendar events, health data, etc. These shouldn’t need to be anywhere but on-device. 3. Allow on-device AI to use shortcuts(?)

I think most of the functionality are well served on desktop front with Ollama and LM Studio, but moving these functionality to mobile offers a great learning opportunity.